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Many  lines  of  investigation  implicate  the  right  hemisphere  in  the 
comprehension  and  production  of  emotional  prosodic  information,  independent  of  the 
valence  of  the  prosody,  supporting  a right  hemisphere  dominance  hypothesis.  An 
alternative  valence  hypothesis  suggests  that  the  production  of  positive  prosody 
preferentially  activates  left  hemisphere  regions  and  predicts  that  the  production  of 
negative  prosody  will  produce  greater  right  hemisphere  brain  activity.  Functional 
magnetic  resonance  imaging  (fMRI)  was  utilized  to  test  these  alternative  hypotheses 
and  learn  more  about  the  anatomic  substrate  of  emotional  prosodic  production.  This 
recently  developed  technique  is  able  to  reveal  anatomic-functional  relationships.  The 
brain  activity  of  twelve  normal  subjects  while  expressing  emotional  prosody  was 
examined  for  right  versus  left  hemisphere  differences. 


The  results  of  this  fMRI  study  provided  supportive  evidence  of  the  validity  of 
the  right  hemisphere  dominance  hypothesis.  It  revealed  that  cerebral  activity  was 
greater  in  the  right  hemisphere  than  in  the  left  during  the  expression  of  emotional 
prosody  with  either  positive  (e.g.,  happy)  or  negative  valence  (e.g.,  angry)  when 
compared  with  that  created  during  the  expression  of  neutral  prosody.  These  findings 
provide  evidence  that  the  right  hemisphere  has  a critical  role  in  the  production  of 
emotional  prosody.  This  study  also  provided  tentative  evidence  that  a prosodic 
output  module  may  be  operative  in  the  right  frontal  region. 
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CHAPTER  1 

INTRODUCTION  AND  STATEMENT  OF  THE  PROBLEM 


Vocal  affective  intonations  play  a significant  role  in  communication,  but  the 
substrate  for  this  capability  is  not  well  understood.  To  explore  these  relationships  and 
to  reduce  the  ambiguity  of  this  topic  it  will  be  helpful  to  make  a distinction  between 
emotional  experiences,  mood,  and  emotional  behaviors.  Emotional  experiences  are 
typically  transient,  internal  states  that  are  difficult  to  quantify,  such  as  being  happy. 

If  these  states  are  more  long  lasting  they  are  frequently  described  as  a mood,  for 
example  being  depressed.  In  contrast,  emotional  behaviors  are  the  external  signs  of 
those  internal  emotional  experiences.  These  emotional  behaviors  are  used  to 
communicate  a person’s  internal  emotional  status  through  facial,  postural,  or  vocal 
intonations. 

Emotional  experiences  can  be  categorized  by  two  main  factors,  valence  and 
arousal.  For  instance,  emotions  such  as  happiness  have  been  described  as  having 
pleasant  or  positive  valence,  whereas  emotions  such  as  anger  are  described  as  being 
unpleasant  or  having  a negative  valence.  Arousal  depicts  the  range  of  excitement  or 
intensity  (high  to  low)  associated  with  a particular  emotional  experience.  It  is 
possible  to  monitor  a number  of  physiological  measures  that  appear  to  be  correlated 
with  emotional  experiences,  such  as  changes  in  the  autonomic  nervous  system 
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affecting  heart  rate,  respiration,  or  changes  in  skin  conductance.  However,  it  is  not 
known  how  closely  tied  these  internal  emotional  experiences  are  correlated  with  those 
quantifiable  changes  in  physiology. 

In  a similar  way,  emotional  behaviors  may  or  may  not  be  correlated  with 
changes  in  physiology  or  alterations  in  the  internal  emotional  experience.  It  is 
impossible  to  objectively  determine  the  true  internal  emotional  state  of  an  individual 
and  it  is  conceivable  that  each  observed  emotional  behavior  contains  a mix  of  a 
genuine  emotional  experience  tied  to  an  observable  physiological  response,  along 
with  a measure  of  acting.  William  Shakespeare  wrote  in  As  you  Like  it,  act  II  verse 
36,  that  “All  the  world 's  a stage,  and  all  the  men  and  women  merely  players.  They 
have  their  exits  and  their  entrances;  And  one  man  in  his  time  plays  many  parts.”  One 
interpretation  of  this  passage  is  that  Shakespeare  proposed  that  social  interactions 
require  the  skillful  utilization  of  a wide  array  of  emotional  behaviors  in  an  ever 
changing  dynamic. 

Within  the  theatrical  community,  two  main  approaches  have  been  utilized  to 
describe  the  process  of  acting  which  involves  a conscious  manipulation  of  those 
emotional  behaviors,  to  elicit  the  audience’s  perception  of  genuine  emotional 
experiences.  Elaborating  on  earlier  development  by  Stanislavsky  (1936),  the 
Strasberg  (1987)  acting  technique  known  as  The  Method  encourages  performers  to 
remember  an  emotional  experience  from  their  past  that  matches  their  character’s 
targeted  emotion.  They  are  trained  to  attempt  to  re-experience  this  emotion, 
incorporating  any  emotion  they  are  currently  feeling,  and  in  the  process  try  to 
temporarily  become  the  character  they  are  playing. 
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In  contrast,  most  current  performers  follow  more  pragmatic  techniques  such 
the  Meisner  Technique  (Meisner  & Longwell,  1987)  which  emphasizes  the 
production  of  behaviors  which  mimic  the  target  emotional  behaviors,  yet  they  do  not 
attempt  to  become  the  character  in  the  process.  During  a performance  it  is 
acknowledged  that  the  actor  may  or  may  not  actually  experience  the  target  emotion, 
but  improvises  and  produces  emotional  behaviors  that  are  appropriate  to  the 
emotional  displays  of  the  other  actors.  The  contrast  between  these  two  theatrical 
techniques  can  inform  studies  of  emotional  behavior.  For  instance,  one  must  keep  in 
mind  the  rich  sophistication  of  the  human  emotional  repertoire  and  that  these  subtle 
components  are  impossible  to  objectively  differentiate.  Therefore,  while  emotional 
behaviors  may  be  observed  and  quantified  and  at  times  appear  be  correlated  with 
internal  emotional  experiences,  it  remains  impossible  to  prove  an  actual  relationship 
between  the  two.  At  best,  the  existence  of  emotional  experiences  are  inferred,  based 
on  subjective  report  and  measurements  of  emotional  behaviors. 

In  spite  of  this  ambiguity,  some  of  the  most  creative  theoreticians  of  the  19th 
and  20th  century  have  explored  this  intriguing  human  capability  (James,  1 884). 
Charles  Darwin  himself  wrote  that 

to  understand,  as  far  as  is  possible,  the  source  or  origin  of  the  various 
expressions  which  may  be  hourly  seen  on  the  faces  of  the  men  around  us,  ... 
ought  to  possess  much  interest  for  us.  (Darwin,  1872,  p.  360) 

Since  Darwin  (1872),  studies  of  the  neural  substrates  of  emotion  have 
revealed  that  a collection  of  cortical  and  subcortical  structures  known  as  the  limbic 
system  play  an  important  role  in  emotion.  Although  portions  of  the  limbic  system 
have  a crucial  role  in  supporting  emotional  processes,  these  structures  do  not  appear 
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to  have  an  exclusive  role.  Many  believe  that  both  the  cortex  and  the  limbic  system 
have  a role  in  emotion  processing.  Damasio  (1994). has  argued  that  emotion  may 
provide  a degree  of  systematic  integration  which  helps  individuals  interpret  the  multi- 
faceted events  of  everyday  life 

Left  hemisphere  injury  typically  inflicts  obvious  deficits  in  language  functions 
(Broca,  1863)  yet  right  hemisphere  injuries  impose  disabilities  that  are  far  less  readily 
evident.  Right  hemisphere  injuries  impact  subtle  systems  that  control  visuospatial 
abilities  (Benton,  Hannay,  & Varney,  1975),  attention  monitoring  capabilities 
(Heilman  and  Van  Den  Abel,  1980),  and  the  comprehension  and  expression  of 
emotional  prosody  (Heilman,  1994).  Whereas  the  lexical/semantic  aspects  of 
language  are  typically  intact  after  right  hemisphere  injury  in  right  handed  subjects, 
the  suprasegmental  aspects  that  communicate  emotion  in  the  voice,  such  as  prosody, 
are  often  deficient  (Ross,  1981). 

There  is  evidence  (see  Locke  1997  for  a review)  that  the  initial  building 
blocks  of  language  are  prosodic-intonational  based  on  infants’  emotional  bond  to 
their  mother  and  their  attentiveness  to  the  mother’s  facial  expressions  (Locke,  1993) 
and  prosodic  cues  (DeCasper  & Spence,  1986).  For  example,  it  is  known  that  infants 
are  able  to  attend  and  imitate  intonation  contours  in  the  first  six  months  of  life 
(Masataka,  1 992).  Perception  of  these  prosodic  cues  may  form  a basis  for  more 
advanced  analysis  of  segmental  components  of  the  speech  stream  later  in 
development  (Goodsitt,  Morgan,  & Kuhl,  1993;  MacNeilage,  1998). 

Prosody  can  also  be  utilized  to  convey  linguistic  information  (Behrens,  1988). 
However  emotional  prosody,  rather  than  linguistic  prosody,  will  be  the  focus  of  this 
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study.  Deficits  in  the  production  and  comprehension  of  emotional  prosody  have 
been  documented  to  occur  after  right  hemisphere  injury  (Tucker,  Watson,  & Heilman, 

1977;  Ross  and  Mesulam,  1979;  Blonder,  Bowers,  & Heilman,  1991;  Blonder, 

Pickering,  Heath,  Smith,  & Butler,  1995).  These  deficits  in  the  production  of 
emotional  prosody  induce  a serious  functional  disability  and  limit  the  impact  of 
rehabilitative  services  (Cummings,  1997;  Ween,  Alexander,  D'Esposito,  & Roberts, 

1996).  However,  the  brain  networks  mediating  the  production  of  emotional  prosody 
have  not  been  entirely  elucidated.  Thus,  the  purpose  of  the  present  study  was  to 
reveal  the  physiological  substrate  of  emotional  prosody  during  speech  production  and 
in  the  process  evaluate  the  extent  to  which  this  complex  system  is  lateralized. 

In  this  chapter  evidence  for  the  neuroanatomical  instantiation  of  emotion 
through  animal  modeling  and  human  lesion  studies  will  be  reviewed.  The  system 
supporting  language  and  speech  production  in  the  human  will  also  be  briefly 
discussed.  Finally  a cognitive  neuropsychological  model  of  emotional  prosody 
expression  will  be  described.  This  review  suggested  a number  of  experimental 
comparisons  which  were  executed  to  evaluate  the  validity  of  the  right  hemisphere 
dominance  in  comparison  to  the  valence  hypothesis. 

The  Emotional  Behavior  of  Prosody 

Hughlings  Jackson  (1932)  noted  that  patients  with  profound  aphasia,  even  after  large 
left  hemisphere  lesions,  could  express  emotions  through  single  words  by  altering  the  pitch  or 
prosody  in  their  voice.  Prosody  is  a nonverbal  or  suprasegmental  feature  of  spoken  language 
which  conveys  various  levels  of  information  to  the  listener.  The  multiple  dimensions  of 
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prosody  may  include  pitch,  intonation,  timing,  stress,  cadence,  tempo,  accent,  pauses, 
loudness,  timbre,  or  melody  that  are  delivered  along  with  the  lexical-semantic  components 
during  language  production. 

Monrad-Krohn  (1947a-b)  suggested  that  there  are  four  different  subtypes  of  prosody: 
(1)  intrinsic  or  linguistic  prosody  defined  as  elemental  melodic  patterns  having  standard 
semantic  connotations,  (2)  intellectual  prosody  defined  as  finely  marked  intonations  that  give 
rise  to  subtle  shades  of  meaning,  (3)  inarticulate  prosody  which  he  described  as  the  use  of 
nonlinguistic  speech  sounds  such  as  grunts,  and  (4)  emotional  prosody,  an  emotional 
behavior  which  injects  affect  into  speech  and  is  the  focus  of  this  review. 

A number  of  studies  have  hypothesized  that  the  right  hemisphere  is  more  adept  at 
processing  the  pitch  attributes  of  intonation  contours,  while  the  left  hemisphere  appears  to 
have  an  advantage  in  decoding  the  temporal  dimensions  of  that  stimuli  (VanLancker  and 
Sidtis,  1992;  Pell  and  Baum,  1997).  For  instance  in  a magnetoencephalography  (MEG)  study 
of  Japanese  pitch  accents,  Imaizumi,  Mori,  Kiritani,  Hosoi,  & Tonoike  (1998)  demonstrated 
that  the  left  hemisphere  is  involved  in  the  interpretation  of  linguistic  information  irrespective 
of  acoustic  cues,  while  the  right  hemisphere  appears  to  be  involved  in  the  interpretation  of 
prosodic  cues.  Additionally  a left  ear  (e.g.,  right  hemispheric  superiority)  advantage  has  been 
demonstrated  for  nonlinguistic  stimuli  such  as  music  (Kimura,  1964).  In  contrast,  right 
handed  individuals  generally  demonstrate  a right  ear  advantage  (e.g.,  left  hemispheric 
superiority)  for  processing  linguistic  information  in  dichotic  listening  tasks  (Studdert- 
Kennedy  and  Shankweiler,  1970).  The  explanation  for  these  hemispheric  biases  typically 
focuses  on  the  more  efficient  processing  of  this  information  by  the  respective  hemisphere. 
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Shapiro  and  Danly  (1985)  used  acoustic  analysis  to  examine  patients  with  right 
hemisphere  damage  and  tested  their  ability  to  produce  happy  or  sad  prosody  in  sentences  that 
were  emotionally  neutral  in  content.  They  found  that  patients  with  right  anterior  lesions 
spoke  with  less  fundamental  frequency  variability  when  compared  to  non-brain  damaged 
individuals,  while  those  with  right  posterior  lesions  produced  increased  fundamental 
frequency  variability  for  both  emotional  and  linguistic  utterances.  When  a group  of  raters 
(Pell,  1999a)  evaluated  the  emotional  prosodic  productions  (sad,  happy,  and  angry)  of  10 
subjects  with  right  hemisphere  lesions,  a substantially  greater  number  of  those  utterances 
were  misidentified  as  lacking  emotional  inflection  (e.g.,  a neutral  response  was  recorded) 
when  compared  to  utterances  produced  by  normal  control  subjects.  Pell  (1999a)  noted  that 
the  right  hemisphere  subjects  demonstrated  subtle  difficulties  in  modulating  the  fundamental 
frequency  of  their  emotional  prosodic  utterances.  For  instance,  the  right  hemisphere  subjects 
appeared  to  have  difficulty  with  emotional  targets  which  diverged  substantially  from  a 
neutral  baseline  for  that  vocal  parameter.  Many  have  argued  that  neutral  prosody  represents 
an  independent  point  along  a prosodic  continuum  which  is  then  modified  through  subtle 
inflection  (Grandour,  Larsen,  Dechongkit,  Ponglorpisit,  & Khunadorn,  1995;  Ladd,  1980). 
Pell  (1999a)  concluded  that  the  right  hemisphere  appears  to  have  an  advantage  in  processing 
prosodic  information  which  may  function  in  a graded  rather  than  in  a categorical  fashion  (see 
also  Blonder  et  al.,  1995). 

Receptive  Prosodic  Deficits 

A number  of  studies  have  provided  further  evidence  of  a preferential  role  of  the  right 
hemisphere  in  emotional  comprehension.  For  instance,  in  1975  Heilman,  Scholes,  and 
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Watson  compared  six  patients  with  right  hemisphere  dysfunction  and  neglect  with  six  left 
hemisphere  patients  in  an  attempt  to  determine  whether  prosodic  comprehension  deficits 
were  preferentially  associated  with  a particular  hemisphere.  They  were  presented  with 
emotionally  neutral,  tape  recorded  sentences  which  utilized  one  of  four  emotional  intonations 
(happy,  sad,  angry,  indifferent).  In  half  of  the  trials,  the  subjects  were  asked  to  judge  the 
emotional  intonation  of  the  speaker.  In  the  other  half  they  were  required  to  judge  the  content 
of  the  message.  Heilman  and  colleagues  (1975)  found  that  both  groups  of  patients  were  able 
to  correctly  identify  the  content  of  the  sentences.  However,  the  right  hemisphere  patients  had 
difficulty  identifying  the  emotional  intonation  of  the  speaker,  while  the  left  hemisphere 
patients  did  not.  This  finding  was  replicated  by  Tucker  et  al.  (1977;  see  also  Bowers, 

Coslett,  Bauer,  Spedie,  & Heilman,  1987;  Darby,  1993)  which  supported  a role  of  the  right 
hemisphere  in  the  comprehension  of  emotional  prosody.  Barrett,  Crucian,  Raymer,  & 
Heilman  (1999)  examined  a young  woman  whose  left  hemisphere  had  been  almost  entirely 
destroyed  following  a left  carotid  artery  dissection.  After  this  large  left  hemisphere  injury 
she  was  unable  to  comprehend  even  simple  verbal  commands.  However,  through  a test  of 
her  prosodic  comprehension  with  visual-prosodic  passive  matching  tasks,  it  was 
demonstrated  that  her  comprehension  of  emotional  prosody  was  intact. 

Expressive  Prosodic  Deficits 

While  the  prior  studies  demonstrated  that  right  hemisphere  injury  was  associated  with 
a deficit  in  prosodic  comprehension  (Barrett  et  al.,  1999;  Heilman  et  al.,  1975),  prosodic 
production  was  not  tested  at  that  time.  Clinically  it  has  long  been  known  that  right 
hemispheric  patients  frequently  demonstrate  flat  affect  as  well  as  rushed  speech  in  their 
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verbal  production.  Bell,  Davis,  Morgan-Fisher,  & Ross’s  (1990)  report  of  deficits  in  the 
production  of  prosody  after  right  hemisphere  injury  in  two  school  aged  children  supports  a 
role  of  the  right  hemisphere  in  the  production  of  emotional  prosody.  In  a follow-up  to  their 
original  study  Tucker,  Watson  and  Heilman  (1977)  examined  eleven  patients  with  right 
hemispheric  injuries  and  compared  their  performance  on  tasks  of  prosodic  comprehension 
and  production  with  seven  left  hemispheric  patients.  They  documented  a significant  deficit 
in  prosodic  production  among  the  right  hemispheric  patients  and  argued  that  this  deficit  was 
a result  of  lack  of  access  or  a damaged  prosodic  representation  (see  also  Heilman,  Bowers, 
Speedie,  & Coslett,  1984).  Ross  and  Mesulam  (1979)  described  two  patients  with  a selective 
deficits  in  prosodic  production.  Based  on  these  findings,  and  in  agreement  with  Heilman’s 
earlier  work  (1975),  Ross  and  Mesulam  (1979)  posited  that  the  right  hemisphere  was 
dominant  for  the  coordination  of  prosodic  information  and  suggested  these  prosodic 
representations  are  stored  in  this  hemisphere. 

Explanatory  Theories  of  Prosodic  Deficits 
The  Right  Hemisphere  Dominance  Hypothesis 

Ross  (1981,  1988)  continued  to  study  patients  with  prosodic  deficits  and  further 
explored  the  instantiation  of  prosodic  information  within  the  right  hemisphere.  He  studied 
ten  patients  with  right  hemispheric  injury  and  examined  them  qualitatively  for  (1) 
spontaneous  use  of  affective  prosody  during  conversation,  (2)  their  ability  to  repeat  verbally 
neutral  sentences  with  affective  prosody,  (3)  their  ability  to  auditorily  comprehend  affective 
prosody,  and  (4)  their  ability  to  visually  comprehend  gestures.  He  found  that  the  spectrum  of 
aprosodias  after  right  hemispheric  injury  mirrored  the  attributes  of  language  deficits  with  left 
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hemispheric  patients.  In  particular  he  found  the  following  aprosodias  after  right  hemisphere 
injury:  (1)  motor  aprosodia  after  damage  to  the  right  inferior  frontoparietal  region 
characterized  by  poor  spontaneous  use  of  emotional  prosody,  poor  prosodic  repetition,  and 
good  comprehension  of  prosody;  (2)  sensory  aprosodia  after  damage  to  the  right  posterior 
superior  temporal  lobe  and  posterior  inferior  parietal  lobe  characterized  by  spared 
spontaneous  prosody  in  conjunction  with  poor  prosodic  repetition,  and  poor  comprehension 
of  emotional  prosody;  (3)  global  aprosodia  after  damage  to  right  frontal,  parietal,  anterior 
occipital,  superior  temporal  lobes  characterized  by  poor  performance  in  spontaneous 
prosodic  production,  repetition  and  comprehension;  and  (4)  mixed  transcortical  after 
watershed  lesions  involving  the  right  frontal  and  parietal  lobes  extending  into  the  superior 
temporal  lobe  marked  by  intact  prosodic  repetition  with  poor  spontaneous  prosodic 
production  and  comprehension. 

A cognitive  neuropsychological  model  of  language  production  suggested  by 
Caramazza  (1996)  posits  a hypothesized  input  lexicon,  or  module,  forming  a memory  store 
of  words,  a semantic  system  forming  a storage  network  for  the  meaning  of  words,  and  an 
output  lexicon  or  module  which  contains  the  motor  behaviors  necessary  to  produce  a word 
verbally.  Ross  (1981,  1988)  conjectured  that  an  analogous  system  with  separate  input  and 
output  prosodic  modules,  as  well  as  an  emotional  semantic  system,  may  also  exist.  While 
controversial,  this  model  provides  a conceptual  framework  which  has  permitted  the 
exploration  of  a variety  of  receptive  and  expressive  prosodic  deficits  across  a broad  spectrum 
of  patients. 

A number  of  critiques  could  be  applied  to  Ross’s  hypothesis  (1981,  1988). 

For  instance,  the  potential  role  of  the  basal  ganglia  in  supporting  the  production  of 
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emotional  prosody  is  underdeveloped  in  his  hypothesis  and  his  lesion  studies  were  not 
examined  in  detail  for  damage  deep  to  the  cortical  surface.  Ross  (1981,  1988)  relied 
on  lower  resolution  computed  topographic  (CT)  images  yet  more  recent  and  detailed 
CT  studies  (Starkstein,  Federoff,  Price,  Leiguarda,  & Robinson,  1994)  suggest  that 
patients  with  receptive  prosodic  deficits  have  a higher  frequency  of  right  hemisphere 
lesions  which  also  involved  the  basal  ganglia.  Blonder,  Gar,  and  Gur  (1989)  revealed 
that  patients  with  both  right  and  left  hemiparkinsonism  have  deficits  in  producing 
emotional  prosody,  suggesting  that  the  neural  control  on  intonation  might  be  bilateral 
at  the  level  of  the  basal  ganglia.  Further  evidence  is  provided  by  ablation  studies  of 
non-human  primates.  There  species-specific  calls,  which  resemble  human  emotional 
prosody  acoustic  structure,  appear  to  be  mediated  by  bilateral,  subcortical  brain 
regions  (Jurgens,  1979).  Support  for  the  involvement  of  the  basal  ganglia  in  prosodic 
processing  also  comes  from  studies  of  patients  with  either  Parkinson’s  (Lloyd,  1999) 
or  Huntington’s  disease  (Speedie,  Blake,  Folstein,  Bowers,  & Heilman,  1990)  which 
both  demonstrated  reduced  prosodic  modulation  of  speech.  Another  critique  that 
could  be  applied  to  Ross’s  (1981,  1988)  theory  relates  to  his  conjecture  that  the  right 
posterior  region  is  prominently  involved  in  prosodic  comprehension,  this  hypothesis 
has  not  been  supported  by  later  studies.  For  instance,  George  et  al.  (1996)  provide  in 
vivo  evidence  in  a PET  study  that  comprehension  of  emotional  prosody  activates 
anterior,  not  posterior,  regions  of  the  right  hemisphere. 
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The  Valence  Hypothesis 

An  alternative  to  the  right  hemisphere  dominance  theory  is  the  valence 
hypothesis.  This  posits  that  an  individual  emotion  is  supported  by  a particular 
hemisphere.  Many  have  argued  that  the  left  hemisphere  mediates  positive  emotions 
while  the  right  hemisphere  supports  negative  emotions  (Canli,  Desmond,  Zhao, 
Glover,  & Gabrieli,  1998;  Sackeim  et  al.,  1982).  This  is  based  on  studies  such  as 
Starkstein,  Federoff,  Price,  Leiguarda,  & Robinson  (1987)  which  revealed  that 
patients  with  left  hemisphere  lesions,  especially  left  frontal  ones,  are  often  sad, 
whereas  those  with  right  hemispheric  injuries  are  often  indifferent.  Goldstein  (1948) 
called  the  depression  associated  with  left  hemisphere  lesions  a catastrophic  reaction. 
This  change  in  mood,  most  commonly  associated  with  left,  but  not  right  hemisphere 
lesions,  may  occur  because  the  right  hemisphere  patients  are  not  exposed  to  the 
obvious  deficits  which  left  hemisphere  patients  typically  experience  such  as  an 
aphasia  or  dysfunction  of  the  preferred  hand.  Gainotti  (1972)  proposed  that  right 
hemisphere  patients  experience  a greater  incidence  of  unawareness  of  their  deficits 
(anosognosia),  based  on  their  failure  to  perceive  a superficially  obvious  handicap. 

However,  Milner  (1974)  did  not  find  a greater  incidence  of  depression  in 
patients  with  left  hemisphere  lesions,  and  a study  by  House,  Dennis,  Warlow, 
Hawton,  & Molyneux  (1990)  also  failed  to  replicate  Starkstein  et  al.  (1987) 
asymmetrical  findings. 

Canli  et  al.  (1998)  provided  in  vivo  evidence  for  the  valence  hypothesis  in  a 
fMRI  study  involving  passive  viewing  of  pictures  with  either  positive  or  negative 
valence,  and  reported  that  viewing  pictures  with  positive  valence  elicited  preferential 
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left  hemisphere  activity,  while  pictures  with  negative  valence  elicited  greater  activity 
in  the  right  hemisphere.  Other  imaging  studies  have  supported  the  valence  hypothesis 
and  revealed  differential  lateralization  patterns  when  negative  (right  frontal)  emotions 
were  compared  to  positive  (left  frontal)  emotions  (Imaizumi  et  al.,  1997;  Irwin  et  al., 
1997;  Paradiso  et  al.,  1997;  Sutton,  Davidson,  Donzella,  Irwin,  & Dottl,  1997).  It 
remains  possible  that  the  task  designs  of  these  experiments  failed  to  elicit  the  target 
emotions.  Others  have  hypothesized  that  there  is  no  evolutionary  rationale  for  a 
neural  instantiation  of  those  individual  emotions  (MacNeilage,  1998). 

Possible  Neuroanatomic  Substrates  of  Emotional  Experiences 

While  this  study  will  focus  on  the  hemispheric  brain  laterality  of  the 
production  of  emotional  prosody,  it  is  important  to  briefly  review  the  competing 
models  of  emotion  experience  which  inform  this  discussion.  Emotional  experience  is 
a complex  and  multi-faceted  human  ability  and  many  regions  of  the  brain  have  been 
proposed  as  supporting  these  capabilities.  Many  neuroanatomic  models  have  been 
suggested  over  the  past  100  years,  some  have  been  discarded  while  others  have  been 
re-defined. 

The  Papez  Circuit 

In  1937  Papez  published  his  influential  paper  titled  “A  proposed  mechanism 
of  emotion”  postulating  a central  role  of  the  hypothalamus  in  emotion.  This  was 
based  on  Bard’s  (1934)  earlier  finding  that  the  hypothalamus  was  involved  in  the 
mediation  of  rage  responses.  Papez  (1937)  also  postulated  a role  for  the  hippocampus. 
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mammillary  bodies,  fornix,  anterior  nuclei  of  the  thalamus  and  the  cingulate  gyrus  in 
this  circuit,  because  of  their  connections  from  the  cortex  to  the  hypothalamus. 
However,  later  investigations  have  demonstrated  that  the  Papez  circuit,  as  it  has  come 
to  be  called,  is  primarily  involved  with  memory.  For  instance,  a severe  impairment  in 
new  learning  was  demonstrated  after  bilateral  removal  of  the  medial  temporal  lobe 
region  (Scoville,  1954;  Scoville  and  Milner,  1957).  Later  studies  (Rempel-Clower, 
Zola-Morgan,  & Squire,  1995;  Zola-Morgan,  Squire,  & Amaral,  1986)  demonstrated 
that  even  a small  bilateral  lesion  in  the  CAl(e.g.,  cornu  ammonis)  of  the  hippocampus 
can  induce  a marked  anterograde  memory  impairment,  without  alterations  in  the 
patient's  emotional  experiences  or  behaviors. 

The  Basolateral  Circuit 

Yakovlev  (1948)  suggested  that  a basolateral  circuit  was  an  important 
component  of  the  system  supporting  the  expression  of  emotional  experience. 
Yakovlev  (1948)  proposed  that  this  circuit  included  the  orbital  frontal  and  insular 
cortex,  the  uncinate  fasciculus,  the  anterior  temporal  cortex,  the  amygdala  and 
dorsomedial  nucleus  of  the  thalamus  that  projects  back  to  the  orbito-frontal  cortex. 
Earlier  Kluver  and  Bucy  (1937)  had  shown  that  a bilateral  lesion  of  the  anterior 
temporal  lobe,  including  the  amygdala,  induced  placidity  in  usually  aggressive  rhesus 
monkeys.  Later  Akert,  Greusenr,  Woosley,  & Meyer  (1961)  demonstrated  that  the 
key  element  in  eliminating  aggressive  behavior  was  the  ablation  of  the  amygdala, 
rather  than  the  removal  of  temporal  lobe  neocortex.  Further  evidence  in  support  of 
Yakolev’s  (1948)  proposal  came  from  the  finding  that  stimulation  of  the  amygdala 
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induces  rage  (Ursin,  1960;  Wood,  1956)  and  bilateral  lesions  of  the  amygdala 
impaired  recognition  of  fear  (Scott  et  al .,  1997). 

It  is  tempting  to  conclude  that  the  human  emotional  system  mirrors  that  of 
non-human  primates.  For  instance  in  humans  and  in  primates  there  is  evidence  that 
the  amygdala  has  an  important  role  in  fear  (LeDoux,  1993).  There  are  clinical  reports 
of  humans  who  sustained  bilateral  damage  to  the  amygdala  and  anterior  tempro- 
cortex  which  induced  the  Kluver-Bucy  syndrome  with  blunted  emotional  reactivity, 
hypersexuality,  hyperorality,  and  visual  agnosia  (Lilly,  Cummings,  Benson,  & 

Frankel,  1983).  Additionally  impaired  comprehension  of  facial  emotions  has  also 
been  reported  after  bilateral  amygdala  damage  (Adolphs,  Tranel,  Damasio,  & 

Damasio,  1994  & 1995). 

Whereas  the  amygdala  supports  reflexive  fear  reactions  in  animals,  at  issue  is 
the  extent  to  which  this  reaction  becomes  modulated  in  humans  who  possess  more 
advanced  systems  that  shape  this  primitive  reaction.  Halgren  (1992)  posits  that 
because  higher  level  cognitive  inputs  are  sent  to  the  amygdala,  the  amygdala  may 
actually  have  a less  crucial  role  in  human  emotional  experiences  than  it  does  in 
animals.  The  degree  to  which  cortical  resources  are  engaged  in  emotional 
experiences  and  the  extent  to  which  they  modulate  more  ancient  brain  networks  is  an 
important  question.  Many  lines  of  inquiry  support  the  view  that  wide  expanses  of  the 
cortex  interpret  limbic  system  output,  with  suggestions  that  the  right  hemisphere  has  a 
prominent  role  in  this  dynamic  (Ross,  1981;  Cancelliere  and  Kertesz,  1990;  Borod, 
Andelman,  Obler,  Tweedy,  & Welkowitz,  1992). 
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Heilman’s  Modular  Theory'  of  Emotional  Experiences  and  Behaviors 

Following  a dimensional  approach  to  emotions  pioneered  by  Wundt  (1903), 
Heilman  argued  that  each  emotional  experience  can  be  described  through  an 
understanding  of  three  networks  operating  in  tandem  in  the  brain.  Heilman  (1994) 
proposed  that  these  three  widely  distributed  networks  support  the  conscious 
experience  of  emotion  with  one  network  processing  valence , a second  controlling 
arousal,  and  a third  network  mediating  motor  activation  with  either  approach  or 
avoidance  behaviors  (see  also  Denny-Brown  & Chambers,  1958).  He  argues  that  all 
three  of  these  networks  must  be  taken  into  account  to  explain  a particular  emotional 
experience  of  an  individual.  Heilman  (1994)  suggested  that  the  different  states  of 
these  three  networks  operating  concurrently  is  the  best  model  to  understand  the 
variety  of  emotional  experiences  of  which  humans  are  capable.  It  is  beyond  the  scope 
of  this  text  to  discuss  this  theory  in  detail,  but  Heilman’s  (1994)  overall  framework 
will  be  briefly  reviewed  in  light  of  its  potential  contribution  to  a discussion  of  the 
neuroanatomic  correlates  of  one  emotional  behavior,  emotional  prosody. 

Heilman’s  Valence  Network 

Heilman  ( 1 994)  has  suggested  that  while  neurotransmitter  systems  may 
influence  mood,  he  conjectures  that  moods  and  emotions  may  not  be  equivalent 
emotional  experiences.  He  posits  that  the  sadness  that  has  been  noted  to  occur  after 
left  hemisphere  injury  may  actually  be  a result  of  a modification  of  these 
neurotransmitter  systems.  Starkstein  et  al.  (1987)  provided  evidence  that  this  sadness 
was  associated  with  lesions  in  the  caudate.  They  found  that  the  nearer  to  the  frontal 
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pole  the  lesion  was  located,  regardless  of  the  hemisphere,  the  more  severe  was  the 
change  in  mood.  Tucker  and  Williamson  (1984)  suggested  that  these  hemispheric 
valence  asymmetries  may  be  related  to  asymmetrical  control  of  neuropharmacologic 
systems.  For  instance  they  provided  evidence  that  the  left  hemisphere  is  more 
cholinergic  and  dopaminergic  (see  also  Louilot  and  Le  Moal,  1994;  Sullivan  and 
Gratton,  1 994),  while  the  right  hemisphere  appears  to  be  more  noradrenergic  than  the 
left.  Robinson  and  Starkstein  (1989)  reported  that  pharmacological  changes  after 
stroke  influence  the  incidence  of  depression.  Using  positron  emission  topography 
(PET)  they  reported  that  right  hemispheric  strokes  increased  serotonergic  receptor 
binding  while  left  hemisphere  strokes  induced  less  pronounced  serotonergic  binding. 
Heilman  (1994)  concluded  that  while  these  neurotransmitter  systems  may  influence 
mood  and  valence,  the  underlying  mechanism  of  this  process  remains  unknown. 

Heilman’s  Arousal  Network 

Heilman  (1994)  also  posited  that  emotional  experiences  are  modulated  by  the 
general  arousal  and  motivational  state  of  the  individual.  Perceiving  and  responding  to 
stimuli  require  that  the  organism  be  awake  and  alert.  The  system  supporting  arousal 
ranges  from  the  modular  cortical  limbic  reticular  network  (Mesulam,  1981;  Watson, 
Valenstein,  & Heilman,  1981)  supporting  basic  consciousness,  to  long  range  planning 
of  goal  directed  behaviors  by  the  complex,  but  poorly  understood  dynamics  of  the 


frontal  cortices. 
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Heilman’s  Motor  Activation  Network 

Emotions  are  not  simply  passively  experienced  but  often  elicit  an  active 
behavioral  response.  Heilman  (1994)  proposed  that  the  motor  preparatory  system, 
which  is  engaged  when  subjects  intend  to  act,  is  mediated  by  a widely  distributed 
network  guided  by  the  dorsolateral  frontal  lobe.  This  system  receives  input  from  the 
limbic  system,  posterior  polymodal  association  areas  and  the  premotor  areas.  Lesions 
of  components  of  this  network  may  all  cause  akinesia  which  support  its  role  in  motor 
activation  (Watson,  Miller,  & Heilman,  1978).  Coslett  and  Heilman  (1989) 
demonstrated  that  right  hemisphere  lesions  are  more  likely  to  be  associated  with 
contralateral  akinesia  than  left  hemisphere  lesions.  Furthermore  Heilman  (1994) 
posited  a preferential  role  for  the  right  hemisphere  in  intention  (Coslett  and  Heilman, 
1989).  Patients  with  parietal  lesions,  especially  right  sided  lesions,  may  fail  to  move 
or  move  slowly  in  the  space  opposite  the  parietal  lesion. 

Heilman  (1994)  argues  that  the  right  hemisphere  has  a preferential  role  in  the 
expression  of  emotional  behaviors.  While  the  three  networks  Heilman  (1994) 
discusses  appear  sufficient  to  depict  the  global,  over-arching  systems  supporting 
emotional  experiences  and  some  aspects  of  emotional  behavior,  his  proposal  does  not 
specifically  address  the  mechanisms  which  facilitate  the  verbal  expression  of 
emotion.  It  will  be  beneficial  to  examine  this  intricate  system  in  more  detail,  in  light 
of  its  involvement  in  prosodic  production. 
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Verbal  Expression 

As  noted  earlier  there  are  multiple  modalities  through  which  humans  express 
emotion.  Facial  expressions  were  detailed  in  Darwin’s  (1872)  investigations,  but 
verbal  expression  is  also  used  extensively  in  social  interactions  to  express  internal 
emotional  experiences.  The  process  of  speaking  involves  multiple  systems  which 
have  been  shown  to  fractionate  after  neurological  injury  (Caramazza,  1996).  These 
multiple  components  include  the  motivation  to  speak  or  intentionality  as  elaborated  in 
Heilman’s  (1994)  conceptual  framework,  the  coordination  of  the  oral  musculature, 
on-line  construction  of  lexical/semantic  components  into  phrase  length  segments  and 
the  process  by  which  prosodic  nuance  is  superimposed  on  this  lexical/semantic 
content  to  express  the  emotional  status  of  that  individual.  It  will  be  helpful  to 
examine  these  different  aspects  of  verbal  expression  separately. 

Animal  Models  of  Vocal  Expression 

Only  man  utilizes  spoken  language  and  as  such  examining  vocal  behavior, 
even  among  the  higher  primates,  is  not  an  equivalent  process.  However,  detailed 
experimental  manipulations  with  selective  lesions  and  pathway  tracings  is  ethically 
permissible  with  these  higher  mammals  and  this  has  revealed  a great  deal  about  the 
neuroanatomic  basis  of  the  motoric  aspects  of  vocal  production. 

From  a review  of  this  animal  literature,  Farley  (1989)  argues  that  there  are 
four  main  motor  systems  that  control  vocal  tract  motor  neurons  in  the  human.  The 
first  is  a respiratory  control  system  which  maintains  physiological  blood-gas  levels. 
The  second  is  an  vocalization  system  which  may  also  have  an  emotional  dimension. 
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This  utilizes  portions  of  the  anterior  cingulate  and  the  periaqueductal  gray  area. 
Adametz  and  O’Leary  (1959)  reported  that  periaqueductal  lesions  in  cats  induced 
mutism  and  Jurgens  and  Pratt  (1979a)  revealed  that  lesions  in  the  periaqueductal  gray 
area  abolished  species-specific  calls  in  squirrel  monkeys  (see  also  Bandler  & Shipley, 
1994).  In  non-humans  it  appears  that  this  vocalization  system  supports  calls  of 
distress,  which  can  be  produced  even  in  decerebrate  animals  (Carrive,  Dampney,  & 
Bandler,  1987).  Third  Farley  (1989)  discusses  a general  motor  system  involving  the 
basal  ganglia,  motor  thalamus,  cerebellum,  and  cortical  motor  systems.  Finally,  he 
adds  higher  order  cortical  control  centers  which  are  operative  only  in  humans,  such  as 
Wernicke’s  and  Broca’s  areas. 


The  Frontal  Lobes 
The  Anatomy  of  the  Frontal  Lobes 

The  frontal  lobes  are  a complex,  but  not  well  understood  portion  of  the  brain 
which  receive  a rich  and  complex  assortment  of  inputs  from  a variety  of  other  brain 
regions  (see  Damasio  and  Anderson.  1993  for  a review).  Within  each  hemisphere 
three  main  frontal  regions  have  been  delineated:  (1)  the  precentral  cortex 
(Brodmann’s  area  4),  a thin  ribbon  of  tissue  just  anterior  to  the  rolandic  fissure,  (2) 
the  premotor  region,  encompassing  a larger  area  anterior  to  the  motor  strip  including 
lateral  and  medial  regions  (Brodmann  area  6),  and  (3)  the  vast  prefrontal  region 
anterior  to  the  premotor  cortex  (Brodmann  areas  8-12,  46,  and  47)  including  limbic 
cortices  as  well  (Brodmann’s  areas  24,  25,  and  32). 
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Layers  II  and  III  of  the  precentral  cortex  contains  large  Betz  cells  whose 
axons  descend  to  the  spinal  cord.  However  only  half  of  the  corticospinal  fibers 
originate  in  this  region,  with  the  other  half  coming  from  the  premotor  and 
supplementary  motor  areas  (SMA)  (Pandya  and  Barnes,  1987).  This  well  organized 
projection  is  in  contrast  to  the  multi-faceted  network  which  engages  the  prefrontal 
region.  This  area,  which  expanded  dramatically  during  mammalian  evolution 
(Sanides,  1969),  has  direct  or  indirect  interconnections  to  all  the  other  cerebral  lobes 
via  long  association  bundles.  It  also  receives  input  from  varied  sources  such  as  (1) 
the  limbic  system,  (2)  the  reticular  system,  (3)  the  hypothalamus,  and  (4) 
neurotransmitter  systems  such  as  the  ascending  noradrenergic  fibers  originating  from 
the  locus  ceruleus  located  in  the  pons.  This  prefrontal  region  is  the  only  known  area 
of  cortex  to  receive  strong  projections  from  all  of  these  regions  (Stuss  and  Benson, 
1986). 

The  Function  of  the  Frontal  Lobes 

The  function  of  the  precentral  area  (Brodmann’s  area  4)  is  strongly  associated 
with  the  direct  facilitation  of  basic  motoric  functions.  Electric  stimulation  of 
Brodmann  area  4 in  awake  subjects  produces  small  movements  of  one  muscle  or  a 
small  group  of  muscles  on  the  contralateral  side  of  the  body  (see  Penfield  and 
Rasmussen,  1950  for  a review).  However  stimulation  of  the  premotor  area  produces 
movements  of  much  larger  groups  of  muscles,  with  supplemental  motor  area  (SMA, 
Brodmann' s area  6)  stimulation  eliciting  more  complex  movements.  Therefore  the 
premotor  area  has  been  hypothesized  to  be  involved  in  the  coordination  and  planning 
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of  complex  motoric  sequences  (Roland,  Larsen,  Lassen,  and  Skinhoj,  1980). 

However,  while  the  basic  functions  of  the  motor  and  premotor  areas  have  been 
elucidated,  the  function  of  the  prefrontal  region  is  much  more  enigmatic  (Cavada  and 
Schultz,  2000). 

The  prefrontal  region  has  been  further  subdivided  into  (a)  a dorsolateral 
region,  (b)  an  orbitofrontal  region,  and  (c)  a frontal  medial  region.  While  the 
orbitofrontal  region  appears  to  have  a role  in  decision-making  (Bechara,  Damasio, 
Damasio,  & Anderson,  1994;  Damasio,  Tranel,  & Damasio,  1991),  and  inhibition 
(Roberts  & Wallis,  2000),  it  also  appears  to  have  a role  in  the  interpretation  of 
emotional  stimuli  (Bechara,  Damasio,  & Damasio,  2000)  and  has  prominent  limbic 
connections  (Damasio  and  Van  Hoesen,  1983).  Lesions  in  this  orbitofrontal  region 
are  known  to  produce  personality  changes  (Rolls,  1996).  The  dorsolateral  region  of 
the  prefrontal  lobes  appears  to  have  a more  prominent  role  in  cognitive  tasks  such  as 
verbal  fluency  (Benton,  1968),  associative  learning  (Petrides,  Alivisatos,  Evans,  and 
Meyer,  1993),  and  the  inhibition  of  some  behaviors  (Diamond,  1989;  Drewe,  1975; 
Luria,  1966).  While  the  frontal  medial  region  appears  to  be  more  involved  in  motoric 
intention  with  lesions  in  this  region  producing  mutism,  motoric  preservation  or 
impersistence  (Heilman  and  Watson,  1991).  There  is  evidence  that  this  large 
prefrontal  region  is  involved  in  coordinating  complex  behavior  through  these  parallel 
inhibitory  and  excitatory  pathways  to  which  they  are  connected  (Knight,  Staines, 


Swich,  and  Chao,  1999). 


23 


The  Role  of  the  Frontal  Lobes  in  Language  and  Emotional  Processing 

Since  1861  when  Paul  Broca  reported  a patient  whose  speech  was  limited  to  a 
single  syllable  after  a left  posterior  inferior  frontal  lesion,  a preeminent  role  for  the 
production  of  language  has  been  attributed  to  the  left  frontal  region.  Later  studies  of 
other  patients  with  left  hemisphere  lesions  have  continued  to  support  the  hypothesis 
that  multiple  language  capabilities  such  as  syntax,  phonology,  and  lexical  semantics 
are  preferentially  located  in  this  dominant  hemisphere  (Damasio,  1 992). 

Adametz  and  0’ Leary  (1959)  and  Jurgens  and  Pratt  (1979a)  demonstrated  that 
lesions  in  the  supplementary  motor  area  in  the  frontal  cortex  of  animals  induces  the 
cessation  of  species  specific  calls.  Correspondingly,  human  patients  who  have 
experienced  lesions  in  their  left  supplementary  motor  area  have  demonstrated 
difficulty  with  vocal  production,  which  has  been  termed  transcortical  motor  aphasia. 
This  language  deficit  is  defined  by  decreased  initiation  of  spontaneous  speech,  with  a 
very  slow  rate  of  speech  production.  Additionally,  if  their  lesion  was  large  and 
extended  through  the  medial  frontal  area,  a profound  akinetic  mutism  has  been  noted 
to  occur  (Barris  & Schuman,  1953;  Damasio  & Anderson,  1993;  Goldburg,  1985; 
Nielson  and  Jacobs,  1951).  Asa  result  many  attribute  the  initiation  of  language 
production  to  these  regions  (Benson,  1993;  Rothi,  1990).  Many  functional 
neuroimaging  studies  have  also  implicated  the  medial  frontal  cortex  in  word 
generation  tasks  (Crosson  et  al,  1999a;  Friston  et  al.,  1991;  Warburton  et  al. , 1996) 
or  language  monitoring  tasks  (Binder  et  al.,  1997).  Other  studies  have  replicated  the 
classic  lesion  studies  with  neuroimaging  paradigms  and  have  documented  that  the  left 
dorsolateral  frontal  cortex  plays  a role  in  semantic  access  during  a word  generation 
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task  (Crosson  et  al.,  1997;  Frost  et  al.,  1999;  McCarthy,  Blamire,  Rothman,  Gruetter, 
& Shulman,  1993;  Petersen,  Fox,  Posner,  Mintun,  & Raichle,  1988)  and  in  tasks 
which  require  a semantic  decision  (Demb  et  al,  1995;  Gabrieli,  Poldrack,  & 

Desmond,  1998). 

Broca’s  and  Wernicke’s  areas  (Brodmann’s  areas  44/45  and  parts  of  22/39/40 
respectively)  in  the  left  hemisphere  of  normal  right  handed  subjects  are  complex 
cortical  regions  whose  functioning  is  still  not  well  understood  (Caramazza,  1996; 
Damasio,  Grabowski,  Tranel,  Hichwa,  & Damasio,  1996.  Caplan,  Albert,  & Waters, 

1 999).  Many  envision  that  Broca’s  and  Wernicke’s  areas  are  the  nexus  of  multiple 
parallel  distributed  processing  (PDP)  language  networks  (Dell,  1986).  The  classic 
Wemicke-Geschwind  paradigm  posits  their  respective  role  in  speech  production  and 
comprehension,  however  an  evolving  conceptualization  suggests  that  it  is  more  likely 
that  both  of  these  areas  are  two  poles  along  a perisylvian  continuum  which  perform 
incremental  language  processing  tasks  along  its  cortical  expanse  (Anderson  et  al., 
1999)  potentially  even  engaging  widely  separated  brain  regions  in  this  process 
(Damasio  et  al.,  1996;  Mesulam,  1998). 

The  precise  role  of  Broca’s  area  in  language  production  remains  controversial 
(Goodglass,  1993;  Zatorre,  Meyer,  Gjedde,  & Evans,  1996).  For  instance,  there  is 
evidence  (Foundas,  Eure,  Luevano,  & Weinberger,  1998)  that  Broca’s  area  can  be 
fractionated  into  two  functional  areas.  A anterior  part  (Brodmann’s  area  45,  pars 
trianguluaris)  likely  related  to  lexical  retrieval,  and  a posterior  part  (Brodmann’s  area 
44,  pars  opercularis)  with  mainly  motor  articulatory  functions.  In  fact,  Tauber, 
Raymer,  and  Heilman  ( 1 999)  reported  a patient  with  a dominant  frontal  operculum 
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lesion  which  also  damaged  the  adjacent  premotor  and  primary  motor  cortex  and 
demonstrated  symptoms  consistent  with  a transcortical  motor  aphasia.  This  aphasia  is 
more  commonly  associated  with  SMA  and  anterior  cingulate  lesions  (Atkinson, 

1971).  Lecours  and  Lhermitte  (1976)  noted  that  a dominate  opercular  lesion  induced 
a profound  phonetic  disintegration  impairing  communication.  Therefore  this 
evidence  suggests  that  while  speech  production  does  engage  Broca’s  area 
(Brodmann's  areas  44  & 45)  as  classically  described,  it  does  so  in  conjunction  with 
these  associated  premotor  and  primary  motor  cortex  regions  (Wildgrubber, 
Ackermann,  Klose,  Kardatzki,  & Grodd,  1996). 

It  is  clear  from  a vast  body  of  literature  that  the  production  of  lexical/semantic 
aspects  of  language  production  emanate  from  the  left  frontal  region.  As  a 
consequence  any  task  requiring  speech  production  will  engage  this  region,  whether 
prosody  is  produced  in  the  speech  production  or  not.  What  has  not  been  determined 
is  why  the  left  frontal  lobe  is  engaged  during  this  task  and  additionally  what  role  the 
right  frontal  cortices  may  have  during  the  expression  of  emotional  prosody. 

While  the  actual  function  of  these  frontal  regions  are  still  debated,  any  neuro- 
imaging experiment  which  requires  an  overt  verbal  response,  will  undoubtedly  reveal 
brain  activity  due  to  the  engagement  of  what  Pinker  (1994)  has  termed  the  language 
organ.  Additionally  if  the  task  requires  the  production  of  emotional  prosody  this  may 
be  sufficient  to  engage  emotionally  related  cortices  and  to  produce  pitch  changes  in 
the  voice  motorically  related  cortices  will  likely  be  engaged.  The  challenge  is  to 
somehow  isolate  the  activity  due  exclusively  to  prosodic  production,  from  that 


26 


associated  with  language  production  more  generally.  Specific  methods  to  accomplish 
this  goal  will  be  discussed  later  in  this  dissertation. 

The  Neuroimaging  of  Emotion  and  Prosody 

As  noted  earlier,  it  is  impossible  to  objectively  determine  whether  a subject  is 
actually  experiencing  a target  emotion  during  an  experimental  trial.  It  has  also  been 
methodologically  difficult  to  create  experimental  paradigms  which  induce  robust  and 
reproducible  emotional  states.  Additionally  the  technical  instruments  themselves 
impose  their  own  set  of  limitations. 

For  instance,  functional  magnetic  resonance  imaging  (fMRI)  while  having 
greater  temporal  and  spatial  resolution  when  compared  to  positron  emission 
topography  (PET),  has  a low  signal-to-noise  ratio,  is  very  susceptible  to  motion 
artifacts,  and  provides  a noisy  and  distracting  testing  environment.  Typically  fMRI 
studies  have  utilized  a blocked  paradigm  which  requires  the  subject  to  perform  a 
experimental  task,  such  as  finger  tapping,  continuously  and  repeatedly  for  a period  of 
time.  For  instance,  1 5 seconds  of  finger  tapping  is  then  followed  by  a block  of  1 5 
seconds  of  rest  as  a control  comparison  state.  This  control  state  often  has  been  rest, 
though  other  more  active  tasks  have  been  used  as  control  states.  This  cycle  is  then 
repeated  many  more  times  to  increase  the  number  of  trials  and  hence  increase  the 
statistical  power  of  the  analysis.  Then  each  individual  voxel  monitored  during  the 
experimental  period  is  compared  to  the  same  voxel  also  monitored  during  the  control 
period.  Those  voxels  that  are  significantly  different  between  the  experimental  and 
control  blocks  are  then  colorized  and  overlaid  onto  a structural  MRI  image. 
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The  majority  of  fMRI  studies  have  utilized  this  simple  blocked  paradigm 
approach  with  great  success,  and  this  approach  appears  to  be  sensitive  to  a wide  range 
of  motoric  and  cognitive  tasks.  However,  one  problem  with  blocked  fMRI  studies  is 
that  other  lines  of  inquiry  indicate  that  cognitive  tasks  such  as  reading  or  lexical 
access  (Posner  and  Abdullaev,  1999)  are  swiftly  processed  on  the  scale  of 
milliseconds.  The  blocked  technique  may  not  have  sufficient  temporal  resolution  to 
reveal  these  very  rapid  brain  events.  This  problem  is  compounded  by  an  issue  which 
effects  all  fMRI  studies.  It  is  known  that  the  repeated  trials  (necessary  to  boost  the 
statistical  power  of  the  analysis)  have  the  potential  to  reduce  the  amount  of  brain 
activity  visualized  because  of  habituation  or  the  subject’s  inattention  to  the  target 
behavior. 

Even  with  these  difficulties  a number  of  studies  have  been  reported  which 
have  explored  the  neuroanatomic  correlates  of  emotion  and  prosody.  For  instance, 
Phillips  et  al.  (1997)  reported  the  findings  from  their  fMRI  study  which  revealed 
amygdala  involvement  during  the  perception  of  fearful  faces.  Beauregard  et  al. 

(1997)  in  a PET  study  compared  the  patterns  of  brain  activity  associated  with  the 
passive  viewing  of  words  with  either  neutral  (animal  names)  or  emotional 
connotations.  They  reported  that  the  passive  viewing  of  words  with  emotional 
connotations,  when  subtracted  from  a control  task  of  viewing  plus  signs  (e.g.,  +), 
elicited  brain  activity  in  the  right  frontal  and  midline  orbital  frontal  region. 
Additionally  they  reported  activity  in  the  left  inferior  and  middle  frontal  gyri  arguably 
due  to  the  language  demands  of  the  task.  Maddock  and  Buonocore  (1997)  utilized 
passive  listening  of  threat  related  words  contrasted  with  passive  listening  to  neutral 
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words  in  a blocked  fMRI  experiment  which  revealed  left  posterior  cingulate  gyrus 
activity  in  eight  of  the  10  subjects.  In  a PET  study  Lane  et  al.  (1997)  demonstrated 
that  the  passive  viewing  of  emotionally  evocative  pictures,  contrasted  with  passive 
viewing  of  emotionally  neutral  pictures,  revealed  activity  in  bilateral  prefrontal  area 
(Brodmann’s  area  9)  for  pictures  with  either  positive  or  negative  valence.  In  general, 
these  findings  support  some  role  of  this  prefrontal  region  in  tasks  which  involve 
lexical  access  or  emotional  processing,  but  the  specific  regions  engaged  in  each  task 
are  still  underspecified. 

Crosson  et  al.  (1999b)  in  a fMRI  study  provided  evidence  that  covertly 
generating  words  with  emotional  connotations  engaged  cortex  near  the  left  frontal 
and  temporal  poles,  when  compared  to  generating  emotionally  neutral  words. 

Crosson  and  colleagues  (1999b)  hypothesized  this  occurred  because  these  regions 
were  involved  in  processing  semantic  attributes  of  those  words,  that  is,  their 
emotional  connotation.  It  is  important  to  note  that  due  to  technical  limitations 
Crosson  et  al.  (1999b)  monitored  brain  activity  in  only  the  left,  not  the  right, 
hemisphere  during  these  tasks  and  that  their  emotional  word  generation  task 
, alternated  between  cues  with  positive  as  well  as  negative  valence. 

As  noted  earlier,  Canli  et  al.,  (1998)  examined  activity  in  both  hemispheres  in 
a fMRI  study  involving  passive  viewing  of  pictures  with  positive  emotional  valence 
contrasted  with  pictures  with  negative  valence.  They  reported  that  pictures  with 
positive  valence  preferentially  elicited  left  middle  frontal  gyrus  activity,  as  well  as 
left  middle  and  superior  temporal  gyri.  However,  when  viewing  pictures  with 
negative  valence  Canli  and  colleagues  (1998)  reported  activity  in  the  right  inferior 
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frontal  and  in  the  gyrus  rectus  in  the  orbitofrontal  lobe.  This  provides  some  support 
for  the  earlier  described  valence  hypothesis,  but  it  does  not  address  the  issue  of 
emotional  prosodic  production. 

More  specific  to  the  neuroanatomic  correlates  of  emotional  prosody,  George 
et  al.  (1996)  explored  whether  normal  subjects  would  show  differing  patterns  of  brain 
activity  in  their  right  hemispheres  during  a PET  study  while  they  were  engaged  in 
comprehending  emotional  prosody  with  either  positive  or  negative  valence.  They 
found  that  listening  to  emotional  prosody  (with  either  happy,  sad,  or  angry  prosody) 
preferentially  activated  the  right  prefrontal  cortex,  while  comprehending  neutral 
declarative  sentences  activated  the  prefrontal  cortex  bilaterally,  on  the  left  more  so 
than  on  the  right.  Furthermore  Pihan,  Altenmuller,  & Ackermann,  (1997)  in  an  EEG 
study  with  16  normal  subjects  also  reported  that  when  the  subjects  listened  to 
emotional  prosody,  a significantly  lateralized  negativity  towards  the  right  hemisphere 
was  revealed  in  the  EEG  signal.  It  is  important  to  note  that  the  George  et  al.  (1996) 
findings  involved  averaged  functional  data  across  his  13  subjects,  and  that  the  spatial 
resolution  of  PET  and  the  EEG  technique  used  in  the  Pihan  et  al.  (1997)  study  was 
quite  limited. 

No  previous  fMRI  studies  have  examined  the  neuroanatomic  correlates  of  the 
overt  production  of  emotional  prosody.  One  factor  is  that  overt  speech  production 
easily  induces  movement  artifacts  in  fMRI  studies.  Flowever  recently  developed 
fMRI  techniques  which  utilize  event-related  techniques  (Birn  et  al.,  1999a-b;  Buckner 
et  al.,  1996;  Rosen,  Buckner,  & Dale,  1998)  show  promise  of  revealing  the  patterns 
of  brain  activity  associated  with  the  production  of  emotional  prosody  on  the 
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individual  subject  level  across  both  hemispheres.  A more  in-depth  discussion  follows 
in  which  the  technical  aspects  of  the  fMRI  experimental  scans  and  the  rationale  for 
the  use  of  this  particular  neuro-imaging  technique  is  discussed. 

fMRI  - Functional  Magnetic  Resonance  Imaging 

The  neuro-imaging  technique  of  fMRI  provides  a non-invasive  method  to 
explore  brain/behavior  relationships  in  vivo  with  either  normal  subjects  or  subjects 
with  neurological  disease.  While  studies  based  on  patients  with  cerebral  dysfunction 
are  of  great  value,  it  is  also  true  that  the  anatomy  of  the  resulting  damage  often 
depends  more  upon  the  vascular  anatomy  of  the  brain,  rather  than  the  functional 
networks  operating  within  it.  Functional  MRI  provides  a method  by  which  the 
regional  cerebral  blood  oxygenation  related  to  a cognitive  task  can  be  visualized. 

This  is  based  on  the  BOLD  (blood  oxygenation  level  dependent)  technique  related  to 
changes  in  magnetic  susceptibility  related  to  the  volume  of  deoxygenated  blood  in 
task-related  brain  regions  (Frostig,  Lieke,  Ts,  & Grinvald,  1990),.  Since  its  inception 
in  1991  (Belliveau  et  al.,  1991),  fMRI  has  shown  promise  of  depicting  changes  in  the 
relative  volume  of  deoxygenated  blood  volumes  associated  with  cognitive  tasks  in  a 
safe  and  non-intrusive  fashion. 

Early  fMRI  studies  utilized  a blocked  paradigm  (Bandettini,  Jesmanowicz, 
Wong,  & Hyde,  1993)  in  which  the  subject  performed  a task  during  an  active  period, 
for  example  finger  tapping  as  rapidly  as  possible  during  a 15-second  period,  and  then 
they  were  cued  to  lie  quietly  for  the  next  1 5 seconds  during  a rest  period.  This 
active/rest  cycle  was  then  repeated  a number  of  times  in  order  to  improve  the  low 
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signal-to-noise  ratio  of  the  MRI  signal.  Then  during  processing  after  the  scan  had 
been  completed,  comparisons  would  be  made  between  the  active  versus  rest  cycles  in 
individual  voxels  to  visualize  the  additional  brain  activity  that  was  occurring  as  a 
result  of  the  demands  of  finger  tapping.  If  the  subject  had  not  moved  excessively 
during  the  scan,  and  the  computations  had  been  set  up  correctly,  the  result  would 
produce  a map  of  cerebral  brain  activity  that  occurred  during  the  execution  of  the 
finger  tapping  task,  above  and  beyond  the  baseline  activity  associated  with  the  rest 
period  (see  Figure  1). 


Figure  1 . Cerebral  brain  activity  in  the  left  motor  cortex  associated  with  a right  thumb 
to  index  finger  tapping  task  depicted  in  the  axial  plane  from  a preliminary  fMRI 
motoric  experiment  (Anderson  et  al.,  1998). 
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This  blocked  technique  was  initially  utilized  in  tasks  which  activated  primary 
visual  or  motor  cortices  and  this  was  later  expanded  to  examine  more  subtle  cognitive 
paradigms  such  as  higher  order  language,  memory,  or  visuo-spatial  abilities. 
Subsequent  fMRI  statistical  development  has  provided  other  options  such  as 
statistical  parametric  mapping  (SPM;  see  Friston  et  ai,  1995),  motion  correction 
algorithms  (Woods,  Cherry,  & Mazziotta,  1993)  and  event  related  techniques  (Friston 
etal.,  1998). 

Development  of  scanner  technology  has  created  advancements  in  the  speed  of 
acquisition  of  the  MRI  signal  which  has  permitted  larger  volumes  of  the  brain  to  be 
monitored.  For  each  fMRI  protocol,  choices  must  be  made  between  the  spatial 
resolution  that  is  required  for  each  experiment,  the  speed  at  which  a single  image  is 
acquired,  and  the  size  of  the  datasets  that  will  be  created  from  a single  functional 
scan.  The  size  of  the  datasets  is  important  because  while  the  high-end  Unix 
workstations  can  easily  manipulate  this  large  collection  of  files  that  are  typically 
produced  with  a single  subject,  the  older  clinical  MRI  scanners  that  are  utilized  for 
research  often  have  insufficient  memory  for  this  task. 

There  are  now  a number  of  statistical,  computational  approaches  that  an  fMRI 
investigator  may  choose  to  examine  their  data.  One  such  program  developed  by  Cox 
(1996)  at  the  Medical  College  of  Wisconsin  in  Milwaukee,  called  AFNI  (Analysis  of 
Functional  Neuro  Imaging),  is  an  integrated  software  package  (see  Figure  2)  which 
provides  a number  of  statistical  options  for  data  analysis  such  as  a correlational 
comparisons  of  the  change  in  the  MR  signal  with  a reference  wave  or  t-test 
comparisons  between  different  functional  scans  (Bandettini  et  al.,  1993).  For 
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example,  in  the  earlier  discussion  of  a finger  tapping  task,  a sinusoidal  reference  wave 
was  utilized  which  depicted  the  presumed  high  levels  of  activity  in  the  contralateral 
motor  area  associated  with  finger  tapping,  contrasted  with  low  or  baseline  levels  of 
activity  in  the  motor  area  during  the  control  period  when  finger  tapping  was  not 
occurring. 


Figure  2.  Screen  shot  of  the  Unix-based  software  program  AFNI. 
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Figure  3.  Artifactual  cerebral  activity  is  present  in  the  anterior  and  posterior  regions 
of  this  axial  image,  likely  correlated  with  head  movement  during  task  performance. 
This  right  thumb  to  index  finger  tapping  task  also  revealed  probable  task  related 
activity  in  the  motor  cortices  of  the  left  hemisphere  from  this  preliminary  fMRI 
experiment  (Anderson  et  al.,  1998). 


Overt  vs.  Covert  Expression 

Many  neuro-imaging  investigators  have  pursued  language  studies  which 
utilized  covert,  as  opposed  to  overt,  verbal  expression  paradigms  because  of  the 
problem  of  motion  artifact  in  overt  experiments  secondary  to  head  motion  (Friedman 
et  al,  1998;  Phelps,  Hyder.  Blamire,  & Shulman,  1997;  Zelkowicz,  Herbster,  Nebes, 
Mintun,  & Becker,  1998)  or  the  desire  to  eliminate  the  possibility  of  hearing  one’s 
own  voice  (Warburton  et  al.,  1 996).  PET  covert  naming  trials  have  shown  that 
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similar  left  perisylvian  areas  are  activated  when  compared  to  overt  naming  trials 
(Salmelin,  Hari,  Lounasmaa,  & Sams,  1994).  However,  the  covert  trials  demonstrate 
somewhat  decreased  volumes  of  activity  overall,  especially  with  respect  to  activity  in 
the  left  frontal  region.  This  covert  naming  technique  has  facilitated  the  creation  of  a 
large  body  of  studies.  However,  overt  and  covert  behaviors  are  different.  If  subjects 
move  their  head  during  speaking,  even  if  only  slightly,  disruptions  in  the  acquision  of 
the  MRI  signal  and  artifactual  activity  is  produced  (see  Figure  3).  Additionally,  it  is 
known  that  overt  speech,  which  requires  the  movement  of  air  through  air  filled 
cavities  just  inferior  to  the  orbitofrontal  cortex,  may  lead  to  a reduction  in  the  signal- 
to-noise  ratio  through  the  creation  of  regions  of  hyper-  or  hypo-intensity  in  the  raw 
MR  signal  (Birn,  Bandettini,  Cox,  Jesmanowicz,  & Shaker,  1999a;  Birn,  Bandettini, 
& Shaker,  1 999b).  However,  a number  of  techniques  have  been  developed  that  show 
promise  of  overcoming  this  inherent  limitation  of  fMRI. 

T-test  analysis 

For  instance,  it  is  possible  to  utilize  a Student  t-test  analysis  which  would 
compare  activation  from  one  period  or  epoch  of  a trial  when  the  hemodynamic 
response  has  peaked  in  engaged  cortices  to  another  epoch  when  the  hemodynamic 
response  should  have  returned  to  a baseline  level  of  activity.  This  pseudo-event 
related  technique  has  the  potential  of  reducing  the  artifactual  activation  related  to 
speaking  by  eliminating  the  data  gathered  when  the  subject  was  speaking  (see  Figure 
4).  It  would  be  optimal  to  reveal  individual  patterns  of  task-related  brain  activity, 
however  a recent  article  by  Barch  et  al,  (1999)  while  validating  the  use  of  overt 
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speech  production  in  a fMRI  experimental  paradigm,  strongly  recommended  the  use 
of  averaged  group  statistical  analysis  because  of  variable  responses  at  the  individual 
subject  level. 


motion 


Figure  4.  Motion  artifact  occurs  during  speaking  (solid  bars)  during  the  brief  initial 
time  period,  while  the  BOLD  MRI  response  due  to  speaking  evolves  over  a longer 
duration. 


Purpose  of  the  Study  and  Hypotheses 

In  summary,  this  review  provides  evidence  that:  (1)  Emotional  prosody  is  an 
integral  component  of  verbal  expression,  with  the  limbic  system  and  the  frontal  lobes 
playing  an  important  role  (Tucker  et  ai,  1977;  Ross  and  Mesulam,  1977;  Maddock 
and  Buonocore,  1997).  However,  it  is  also  clear  that  cortical  association  areas  also 
modulate  and  interpret  these  emotional  experiences.  It  is  still  controversial  whether 
the  right  hemisphere  has  a preferential  role  in  this  function  as  the  right  hemisphere 
dominant  theory  stipulates.  Alternatively  emotions  may  be  bilaterally  facilitated  as 
the  valence  hypothesis  suggests,  with  the  right  hemisphere  mediating  negative 
emotions  and  the  left  positive  emotions.  (2)  The  neuroanatomic  location  of  the 
systems  supporting  language  and  emotional  prosody  are  complex  and  not  easily 
separated  by  either  ablation  or  neuroimaging  paradigms.  For  instance,  ablation 
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studies  have  provided  the  strongest  evidence  for  the  neuroanatomic  location  of  these 
functions,  but  functional  activity  is  poorly  mapped  across  typical  lesion  distribution 
patterns  which  appear  to  reflect  hemodynamic,  rather  than  functional  factors. 
Initially,  neuroimaging  technologies  such  as  PET  aided  investigations  of  the  specific 
substrates  of  language  and  emotion,  but  the  poor  spatial  resolution  of  this  technique 
has  limited  the  anatomic  precision  of  these  investigations.  The  more  recent  in  vivo 
neuro-imaging  methodology  of  fMRI  shows  the  greatest  promise  of  revealing  these 
anatomic-functional  relationships  with  even  more  specificity  than  PET  or  ablation 
studies.  However,  new  fMRI  approaches  will  need  to  be  utilized  to  address  the 
multiple,  conflicting  demands  of:  (a)  permitting  overt  production  of  speech  prosody 
while  avoiding  motion  induced  artifact,  (b)  providing  sufficient  brain  coverage  to 
permit  comparisons  between  hemispheres  along  with  rapid  acquision  of  MR  images, 
(c)  generating  a sufficient  number  of  trials  to  offset  the  poor  signal-to-noise  ratio  of 
fMRI,  yet  retaining  the  subject’s  attention  throughout  those  multiple  trials  to  avoid 
habituation  or  inattention  resulting  in  a decrease  of  brain  activity.  (3)  From  a 
cognitive  neuropsychological  perspective  it  is  possible  that  a prosodic  output  module 
may  exist  which  can  be  independently  damaged.  However,  the  neuroanatomic 
substrate  for  a hypothesized  system  that  mediates  the  expression  of  emotional 
prosody  has  been  only  tentatively  established.  Based  on  the  preceding  review  a 
number  of  questions  remain  unanswered. 
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Research  Question  1 

Does  the  experimental  evidence  support  the  validity  of  the  right  hemisphere 
dominance  hypothesis,  the  valence  hypothesis,  or  a third  alternative?  That  is,  if  a 
group  of  subjects  express  emotional  prosody  during  overt  verbal  expression  in  a fMRI 
experiment,  will  this  preferentially  activate  the  right  hemisphere  (independent  of  the 
valence  of  the  emotion)  in  support  of  the  right  hemisphere  dominance  hypothesis,  or 
alternatively  will  the  hemispheric  brain  activity  recorded  be  dependent  on  the  valence 
of  the  emotional  prosody  in  support  of  the  valence  hypothesis? 

Research  Question  2 

Is  there  any  evidence  for  the  existence  of  a prosodic  output  module  in  the  right 
frontal  lobe  and  can  a cognitive  model  of  prosody  production  be  generated  from  this 
information?  Ross  (1981)  provided  evidence  that  lesions  in  the  right  frontal  lobe 
induce  deficits  in  the  expression  of  emotional  prosody.  From  a cognitive  model  of 
prosody  production,  it  is  possible  that  a hypothesized  output  module  exists  in  the  right 
frontal  lobe  which  stores  motoric  representations  of  emotional  prosody  activated 
during  verbal  production.  If  this  assumption  was  correct,  greater  activation  of  the 
right  frontal  lobe  would  be  present  during  neuroimaging  tasks  which  required 
prosodic  expression. 

A priori  Prediction  Research  Question  1 

It  was  predicted  that  an  fMRI  experiment  would  provide  greater  support  for 
the  right  hemisphere  dominance  hypothesis  when  the  production  of  emotional 
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prosody  was  compared  to  neutral  prosody  production.  It  was  assumed  that  prosody 
vs.  null  hypothesis  comparisons  would  induce  substantial  left  hemisphere  brain 
activity  as  a result  of  lexical/semantic  demands,  but  these  would  be  minimized  when 
compared  to  neutral  prosody.  This  finding  would  challenge  the  valence  hypothesis 
and  lend  support  to  the  right  hemisphere  dominance  theory. 

A priori  Prediction  Research  Question  2 

It  was  predicted  that  the  results  of  this  experiment  would  reveal  activity  in  the 
right  frontal  lobe  during  the  production  of  emotional  prosody,  which  would  provide 
tentative  evidence  for  this  hypothesized  prosodic  output  module.  This  was  based  on 
ablation  studies  (Ross  & Mesulam,  1979;  Tucker,  Watson,  & Heilman,  1977)  which 
noted  that  right  frontal  lesions  induced  a deficit  in  prosodic  expression.  It  was 
assumed  that  only  limited  evidence  for  a cognitive  model  of  prosody  production 
would  be  provided  by  this  study. 


CHAPTER  2 
METHODOLOGY 

Subject  Screening  Measures 

Twelve  (six  male  and  six  female)  right  handed  subjects  between  the  ages  of 
18  and  45  who  were  recruited  from  the  University  of  Florida  academic  community  to 
take  part  in  this  study.  Subjects  were  screened  prior  to  their  participation  in  the  study 
and  filled  out  a number  of  questionnaires  which  determined:  (1)  handedness,  as 
determined  by  the  revised  Waterloo  Handedness  Questionnaire  (Steenhuis  and 
Bryden,  1989)  and  the  Briggs  & Nebes  Handedness  Inventory  (Briggs  and  Nebes, 
1975);  (2)  languages  spoken  and  age  of  acquisition;  (3)  evidence  of  depression  (Beck 
Depression  Inventory;  Beck,  1987)  and  past  neurological  history;  and  (4)  educational 
background.  Potential  risks  of  the  experiment  were  explained  to  all  subjects  and  each 
gave  written  informed  consent  in  compliance  with  the  University  of  Florida’s  Health 
Center  Institutional  Review  Board  guidelines.  Criteria  for  exclusion  from  this  study 
included:  a less  than  strongly  right  handed  on  the  Waterloo  Handedness 
Questionnaire  bias  (a  score  of  less  than  52  out  of  a possible  72),  the  initial  acquisition 
of  a language  other  than  English,  any  history  of  a psychiatric  disorder  (DSM  IV), 
evidence  of  depression  which  might  limit  a subject  from  expressing  emotional 
prosody,  any  developmental  learning  disorder  including  attention 
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deficits/hyperactivity  disorders  (ADHD),  or  a prior  neurological  injury.  While 
education  was  noted,  it  did  not  serve  as  an  exclusion  criteria. 

The  Waterloo  Handedness  Questionnaire  provided  subjects  with  36  questions 
such  as  “Which  hand  would  you  use  to  adjust  the  volume  knob  on  a radio?”  with  an 
always  right  hand  response  receiving  a +2  score,  a usually  right  hand  response 
receiving  a +1  score,  a either  hand  response  received  a 0 score,  while  a usually  left 
hand  response  received  a -1  score  and  an  always  left  hand  response  received  a -2 
score.  The  scoring  system  for  the  Briggs  and  Nebes  Handedness  Inventory  utilized 
the  same  scoring  system  as  the  Waterloo  Handedness  Questionnaire. 

The  Beck  Depression  Inventory  (1987)  provided  subjects  with  21  groups  of 
four  printed  sentences.  The  task  was  to  choose  one  sentence  among  each  group  of 
four  which  best  depicted  their  emotional  state  over  the  past  week.  One  of  these  four 
potential  responses  was  not  related  to  depressive  symptomology  such  as  “I  am  not 
particularly  discouraged  about  the  future”  while  each  of  the  other  three  in  each  set 
had  an  advancing  degree  of  depressive  symptomology.  The  sentence  without  a 
depressive  tone  was  scored  0,  the  sentence  with  the  mildest  depressive  relatedness 
would  have  received  a score  of +1,  and  so  on.  The  sentence  that  expressed  the 
greatest  degree  of  depression  in  each  set  would  have  received  a score  of  +3. 

After  the  handedness  and  depression  measures  were  completed  and  the  history 
had  been  recorded,  individuals  who  were  admitted  as  study  subjects  performed  the 
three  control  measures  and  the  experimental  tasks  that  day.  Each  control  and 
experimental  task  is  described  separately  below. 


42 


Control  Measures 

Subject  responses  during  the  experimental  tasks,  administered  within  the 
scanner  environment,  could  have  varied  due  to  a number  of  factors  that  were 
unrelated  to  the  target  task  of  producing  emotional  prosody.  Therefore  a number  of 
control  measures  were  utilized  to  reveal  these  potential  sources  of  variability  that 
could  have  influenced  their  performance.  For  instance,  it  is  possible  that  subjects 
would  have  been  distracted  by  the  loud,  confining  environment  of  the  scanner.  This 
environment  could  also  have  induced  anxiety  which  potentially  would  have  limited 
the  extent  to  which  they  could  have  sounded  and  felt  happy.  Therefore  Control 
Measure  1 audio  recorded  the  subject’s  spoken  responses  in  the  scanner  to  allow  the 
opportunity  to  describe  the  prosody  of  each  subject  during  experimental  testing  and 
this  was  later  compared  to  the  subject’s  prosodic  production  outside  of  the  scanner 
environment.  Due  to  the  loud  acoustic  environment  in  the  MR  scanner,  the  subject’s 
actual  prosodic  production  might  still  not  have  been  recordable.  Therefore  the 
subject’s  production  of  emotional  and  neutral  prosody  outside  the  scanner 
environment  was  evaluated  in  free  field  testing. 

Furthermore,  even  if  the  subject’s  production  of  emotional  prosody  matched 
the  target  provided  during  the  experimental  task,  they  might  still  have  actually 
experienced  a different  emotion  during  the  trial.  Therefore  Control  Measure  2 
provided  each  subject  with  a questionnaire  which  they  filled  out  after  the  scan  had 
occurred  to  probe  their  subjective  perception  of  their  prosodic  production  during  the 
experimental  trials.  The  particulars  of  each  of  these  control  measures  are  described 


below. 
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Control  Measure  1:  Prosody  Production  Within  and  Outside  of  Scanner 

During  the  scan,  the  subject’s  responses  were  audiotaped  to  document  the 
subject’s  correct  production  of  the  target  prosody.  To  ensure  a consistent  response 
across  subjects,  each  subject  received  a five  minute  training  about  the  timing  and 
intensity  of  their  verbal  response  while  in  the  scanner  environment.  Subjects  were 
instructed  to  produce  the  spoken  sentence  with  their  normal  conversational  voice  and 
speech  volume.  During  the  production  of  emotional  prosody  subjects  were 
encouraged  to  produce  a level  of  emotional  intensity  that  would  be  natural  for  them  to 
utilize  during  social  conversations.  It  was  suggested  that  they  attempt  to  experience 
the  cued  target  emotion  during  the  experimental  trials.  During  all  trials  subjects  were 
instructed  to  minimize  head  movement  during  speaking. 

To  assess  the  subject’s  performance  outside  of  the  scanner  environment,  18 
trials  of  prosodic  productions  in  free  field  testing  were  also  performed.  Six  trials  each 
of  positive,  negative,  and  neutral  valenced  prosodic  renditions  of  the  same  target 
sentence  used  in  the  scanner  during  experimental  testing,  namely,  “They  went  to  the 
store”  were  recorded  to  assess  the  subject’s  prosodic  accuracy  outside  of  the 
distracting,  anxiety-provoking  environment  of  the  scanner. 

The  in-scanner  recording  was  made  possible  by  a specially  adapted  bi- 
directional dual  microphone  (Jabra  Corp,  San  Diego,  CA)  which  minimized  the 
background  sound  of  the  scanner  by  phase  reversing  the  waveform  associated  with 
this  sound  and  subtracting  it  from  the  speech  and  scanner  signal.  Free  field  testing 
was  recorded  (Sony  model  TCM-323  CE7)  utilizing  the  built-in  microphone  onto 
audiotape  (Maxell  model  C90)  in  a quiet  room  after  the  scanning  session.  The 
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microphone  was  six  inches  from  the  mouth  and  this  was  consistent  across  all  the 
subjects.  These  audiotape  recordings  were  later  reviewed  off-line  by  two  judges  and 
rated  as  to  the  accuracy  of  prosodic  production.  The  judges  were  clinical  speech 
language  pathologists  (one  male,  one  female)  who  were  familiar  with  the  assessment 
of  prosodic  speech  disorders.  The  judges  were  instructed  to  evaluate  each  trial  for 
appropriate  use  of  pitch,  rate,  and  stress,  three  parameters  known  to  characterize 
emotional  prosody  in  verbal  communication.  For  instance,  happy  prosody  is  typically 
produced  at  a fast  rate  with  one  or  two  words  dramatically  stressed  with  sentence 
final  rising  pitch.  However,  prosody  with  negative  valence  is  usually  produced  with  a 
slower  rate  of  production  and  limited  changes  in  pitch,  with  each  word  harshly 
stressed. 

The  judges  were  instructed  to  score  each  subject’s  in  and  out  of  scanner 
prosodic  productions  based  on  the  following  scoring  system  utilized  to  reveal 
gradations  of  prosodic  accuracy.  If  the  judges  assessed  that  the  prosodic  production 
correctly  matched  the  emotional  target  cue  unambiguously  (either  happy,  neutral,  or 
angry  prosody)  they  gave  the  trial  a score  of  +2.  If  the  prosodic  production  of  that 
trial  closely,  but  not  perfectly,  matched  the  target  emotional  prosody  with  less  than 
prototypical  levels  of  pitch,  rate,  or  stress,  it  was  given  a score  of +1.  If  the  subject’s 
prosodic  response  did  not  match  the  cued  emotional  prosody,  a score  of  0 was  given. 

The  two  judges  independently  scored  the  recorded  audiotape  of  Control 
Measure  1 performances  for  all  subjects  and  the  ratings  of  the  two  judges  were  then 
averaged  to  form  a group  average  for  that  trial.  The  averages  for  each  condition 
(happy,  angry,  and  neutral  prosody)  for  each  individual  subject  were  calculated  and  a 
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prosodically  correct  percentage  was  calculated.  For  example,  for  the  25  happy  trials  a 
score  of  50  out  of  a potential  50  would  have  been  given  a 100%  prosodically  accurate 
assessment,  while  a score  of  15  out  of  25  would  have  been  given  a 60%  prosodically 
correct  rating.  The  separate  scores  determined  by  the  two  judges  were  then  compared 
to  each  other  across  all  subjects  to  establish  the  degree  of  agreement  between  these 
two  independent  assessments. 

Control  Measure  2:  Subjective  Evaluation  of  Emotional  Experience 

To  monitor  the  subject’s  self-perception  of  emotional  experience  during  the 
experimental  trials  in  the  scanner,  each  of  the  subjects  were  given  a written 
questionnaire  (see  Appendix  B).  They  were  asked  questions  such  as  “When  you  were 
saying  the  sentence  with  happy  prosody,  did  you  actually  feel  happy,  or  did  your 
voice  only  sound  happy?”,  or  “When  you  were  in  the  scanner  and  asked  to  produce  a 
happy  tone  in  your  voice,  do  you  believe  it  actually  sounded  happy?”. 

Experimental  Tasks 

Along  with  the  required  structural  MR  brain  scans,  three  functional  MRI  scans 
(6  minutes  and  44s  each)  were  obtained  from  each  subject  which  were  composed  of 
25  individual  16.17s  trials  during  which  the  subject  overtly  produced  a single  phrase 
(“They  went  to  the  store”).  While  in  the  scanner,  subjects  were  able  to  hear  the 
experimenter  from  the  control  console  and  task  demands  were  briefly  reviewed  prior 
to  each  individual  scan.  These  instructions  were  provided  by  the  same  individual 
who  provided  the  initial  training  to  ensure  consistent  performance  across  subjects. 
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Two  different  scanner  operators  facilitated  the  technical  aspects  of  the  acquisition  of 
the  MR  signal.  During  each  trial  of  the  functional  scans,  subjects  were  presented  with 
a visual  cue  for  the  target  emotional  prosody  (angry,  happy,  or  neutral).  The 
presentation  of  the  visual  stimuli  (see  Figure  7)  occurred  in  the  initial  2.94s  of  the 
trial  and  the  subject  responded  with  the  overt  production  of  the  target  sentence  with 
the  cued  emotional  prosody  (happy,  angry,  or  neutral).  The  stimulus  display  of  2.94s 
(2  image  epochs)  was  followed  by  a blank  screen  for  an  interval  of  13.23s.  Subjects 
were  instructed  to  lie  quietly  with  their  eyes  open  during  this  13.23s  period.  The 
visually  presented  neutral  sentence  and  target  prosody  (black  text  on  a white 
background)  appeared  on  a screen  that  was  visible  to  the  subject  by  a mirror  built  into 
the  head  coil  of  the  scanner.  The  screen  was  located  fifteen  feet  from  the  subject’s 
head  and  the  letters  were  approximately  eight  inches  in  vertical  height  on  that  screen. 
These  stimuli  were  presented  via  a LCD  projector  connected  to  a Dell  Latitude  XPi 
laptop  running  the  SuperLab  Pro  presentation  program  (Cedrus  Inc.,  Phoenix,  AZ). 

Therefore,  subjects  completed  three  functional  scans  which  were  randomly 
blocked  and  composed  of  25  happy,  25  angry,  and  25  neutral  trials  (16.17s  each  trial). 
Each  functional  scan  lasted  six  minutes  and  44s  in  its  entirety. 

“ They  went  to  the  store ” 

HAPPY 


Figure  5.  An  example  of  the  stimuli  presented  to  subjects  during  the  experiment. 
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fMRI  Scanning  Parameters 

All  studies  were  completed  using  a 1 .5  Tesla  GE  Signa  with  a 2-spiral 
gradient  echo  scan  (Noll,  Cohen,  Meyer,  and  Schneider,  1995)  with  the  following 
parameters:  nine  contiguous  axial  6.9  mm  thick  slices  (1 1 images  per  cycle  or  276 
total  images  per  run),  1 80  mm  FOV,  voxel  size  = 1 .4  x 1 .4  x 6.9  mm  (initial 
acquision),  TR/TE/FA  = 735ms/30ms/40deg,  2.94s  per  image  set,  with  six  initial 
discarded  images  (4.41s).  High-resolution  anatomic  images  were  also  obtained  with 
a 3D-SPGR  sequence  (TR/TE/FA  = 27ms/7ms/45deg),  FOV  = 240  mm,  256  x 256  x 
124  matrix,  voxel  size  = 0.94  x 0.94  x 1.3  mm  in  initial  acquisition).  A domed 
quadrature  rf  head  coil  was  used  with  foam  padding  for  head  stabilization.  Prior  to 
the  functional  images,  time  of  flight  MR  angiograms  were  acquired  for  the  same  nine 
axial  slices  as  the  functional  images. 

fMRI  Data  Analysis 

To  reduce  the  effects  of  head  motion,  images  from  each  run  were  first 
spatially  registered  in-plane  to  a base  image  using  an  iterative  linear  least  squares 
approach  (Keren,  Peleg,  & Brada,  1988).  The  last  image  of  the  last  functional 
imaging  run  was  selected  as  the  base  image  because  its  acquisition  immediately 
preceded  that  of  the  anatomic  images  used  as  an  underlay  for  the  functional  images. 
To  reduce  large  vessel  effects  and  other  potential  sources  of  artifact,  voxels  whose 
standard  deviation  exceeded  5%  of  the  mean  signal  were  set  to  zero  before  further 
processing.  The  individual  time  series  for  each  subject  was  visually  inspected  for 
gross  artifacts. 
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Initially  a prosody  vs.  null  hypothesis  Student  t-test  comparison  was 
performed  on  a voxel-by-voxel  basis  for  each  of  the  three  conditions  (angry,  happy, 
and  neutral).  A very  stringent  statistical  threshold  ( p < 0.000001)  for  positive  voxels 
was  utilized  because  of  the  inherently  high  statistical  power  of  these  initial  gross 
comparisons.  It  was  expected  that  large  volumes  of  bilateral  activity  would  be 
revealed  for  each  of  these  comparisons. 

To  provide  more  specific  experimental  comparisons,  a pseudo-event  related 
technique  that  was  used  in  this  study  was  an  adaptation  of  Birn  et  al.  (1999b) 
technique  which  was  created  to  permit  overt  production  in  a fMRI  experiment.  Each 
trial  was  divided  into  1 1 equal  1 ,47s  epochs  in  a pseudo-event  related  technique  (see 
Figure  6).  During  initial  processing,  a t-test  comparison  was  performed  based  on 
difference  between  the  average  MR  signal  change  produced  during  the  4th  and  5th 
epochs  (4.41s  to  7.35s)  and  the  average  MR  signal  change  created  during  the  last  two 
epochs  (from  13.23s  to  16.17s).  Cohen  (1997)  noted  that  the  hemodynamic  response 
tends  to  peak  in  activated  cortical  tissue  in  this  four  to  seven  second  range.  This 
process  thereby  omitted  from  analysis  the  initial  period  during  which  subjects  spoke. 
This  technique  was  utilized  because  it  had  the  potential  of  reducing  artifactual 
activation  related  to  movement,  thereby  more  clearly  isolating  task  related  activity. 
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Figure  6.  A depiction  of  a single  experimental  trial  during  which  the  subject  sees  the 
visual  stimuli  and  responds  during  the  initial  2 epochs  (2.94s),  the  4th  and  5th  epochs 
which  were  used  as  the  main  basis  of  comparison  (4.41s  to  7.75s),  once  the  baseline 
activation  created  during  the  10th  and  1 1th  epochs  (13.23s  to  16.17s)  were  subtracted. 
Therefore,  only  the  fMRI  data  acquired  during  the  4th  & 5th,  and  10th  & 1 1th  epochs 
were  actually  used  in  analysis. 


The  datasets  of  each  individual  subject  were  then  evaluated  on  a voxel-by- 
voxel  basis  to  compare  the  functional  scans  during  which  emotional  prosody  was 
produced  with  the  scans  during  which  neutral  prosody  was  produced.  A probability 
of  p < 0.001  for  positive  voxels  was  selected  to  provide  a stringent  threshold  for 
significant  brain  activity  and  also  to  eliminate  any  possible  extraneous  artifactual 
activity.  This  Student  t-test  comparison  (p  < 0.001 ; n = 25;  t = 1 .003)  utilized  the 
Cornish-Fisher  expansion  (Fisher  & Cornish,  1960)  to  compensate  for  the  multiple 
comparisons  across  the  12  subjects.  The  averaged  t- values  were  used  to  control  for 
heteroscedasticy  of  MR  signal  variation  between  subjects  because  of  events  unrelated 
to  the  experimental  task  demands  such  as  scanner  variability  between  sessions. 
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variability  in  global  blood  flow  and/or  hematocrit,  or  differing  degrees  of  cardiac 
effects  (Binder  et  al.,  1999). 

After  pre-processing  of  each  of  the  subjects’  datasets  had  been  completed,  all 
of  the  structural  and  functional  datasets  for  each  subject  were  interpolated  to  1 mm3 
voxels,  co-registered  and  converted  to  standard  atlas  space  (Talairach  and  Tournoux, 

1 988)  through  a procedure  within  the  AFNI  program.  To  compensate  for  individual 
subject  variability  in  functional  anatomy,  the  functional  image  volumes  were  spatially 
smoothed  3 mm,  by  replacing  each  voxel  value  with  the  average  value  of  the  others 
within  a 3 mm  radius.  Then  the  functional  datasets  were  merged  across  all  12 
subjects,  by  averaging  the  functional  intensity  represented  by  the  t statistic  for  each 
voxel  across  all  conditions.  A clustering  program  that  is  a component  of  the  AFNI 
package  was  used  to  tabulate  the  number  of  positive  voxels  (1 .0  x 1 .0  x 1.0  mm)  with 
a p value  less  than  0.001  that  were  located  in  a contiguous  cluster  of  at  least  100 
interpolated  voxels  (>  100  micro  liters)  in  volume.  This  information  was  used  to 
evaluate  the  volume  of  significant  activity  for  each  experimental  comparison.  These 
significantly  active  voxels  were  visualized  through  their  display  over  a merged 
structural  image  obtained  from  the  individual  structural  images  of  the  12  subjects. 
Brodmann’s  areas  and  neuroanatomic  gyral  regions  were  determined  by  a 
computerized  database  (Collins,  Holmes,  Peters,  & Evans,  1 995)  which  applied  these 
labels  based  on  the  Talairach  coordinates  (Talairach  and  Tournoux,  1988)  of  the  most 
significant  voxel  within  a single  cluster  of  voxels. 
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Experimental  Comparisons 

Experimental  Comparison  1:  Happy  vs.  Neutral  Prosody 

If  the  right  hemispheric  hypothesis  is  correct,  greater  right  than  left 
hemispheric  engagement  should  have  been  present  as  a result  of  this  comparison.  It 
was  hypothesized  that  the  brain  activity  related  to  the  common  motoric  and  language 
demands  of  both  tasks,  would  have  been  minimized  in  this  process. 

Alternatively,  if  the  valence  hypothesis  is  correct,  this  happy  vs.  neutral 
prosody  comparison  would  predict  preferential  engagement  of  the  left,  not  the  right, 
hemisphere. 

Experimental  Comparison  1 would  provide  the  critical  information  needed  to 
discriminate  between  these  two  competing  models,  because  each  posited  a different 
pattern  of  hemispheric  lateralization  for  this  comparison. 

Experimental  Comparison  2:  Angry  vs.  Neutral  Prosody 

Additionally,  for  each  subject,  a comparison  was  performed  between  the 
activity  related  to  the  expression  of  angry  prosody  and  that  created  during  the 
expression  of  neutral  prosody. 

If  the  right  hemispheric  hypothesis  is  correct,  a angry  vs.  neutral  comparison 
should  have  revealed  greater  right  than  left  hemispheric  engagement.  Similarly,  the 
valence  hypothesis  also  predicted  that  this  comparison  would  have  shown  preferential 
engagement  of  the  right  hemisphere  as  a result  of  this  analysis. 
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Therefore,  while  Experimental  Comparison  2 would  provide  supporting 
evidence  ot  the  validity  of  both  of  these  hypotheses,  it  would  not  have  provided  the 
critical  evidence  needed  to  discriminate  between  these  two  models  which  Comparison 
1 provided. 

Experimental  Comparison  3:  Angry  vs.  Happy  Prosody 

Due  to  the  novelty  of  the  data  analysis  techniques,  two  additional  comparisons 
were  also  performed  to  better  characterize  the  patterns  of  activation  associated  with 
the  other  comparisons.  However,  no  a priori  hypothesis  of  the  results  of  these 
comparisons  was  conjectured,  although  the  results  would  likely  inform  the  analysis  of 
Experimental  Comparisons  1 & 2. 

Experimental  Comparison  4:  Happy  vs.  Angry  Prosody 

An  additional  comparison  contrasted  the  production  of  happy  vs.  angry 


prosody. 


CHAPTER  3 
RESULTS 


Subjects  Screening  Measures 

None  of  the  twelve  subjects  reported  recent  signs  of  depression  or  a past 
history  of  psychiatric  illness  for  themselves  or  their  immediate  family  as  documented 
in  the  pre-scan  subject  checklist  (see  Appendix  1),  and  in  their  responses  to  the  Beck 
Depression  Inventory  (1987).  Each  subject’s  individual  scores  as  well  as  the 
collective  average  scores  for  the  Beck  Depression  Inventory  (mean  of +.75;  SD  = 1.2) 
were  well  below  the  cut  off  score  of +1 1 for  clinical  depression.  Therefore  it  was 
surmised  that  the  subjects  were  not  actively  depressed  at  the  time  of  testing.  The 
subjects  reported  no  past  history  of  neurological  injury,  and  did  not  report  any  history 
of  ADHD  or  other  learning  disabilities. 

The  average  age  of  the  12  subjects  was  27.2  (SD  = 7.4)  years  old  at  the  time 
of  testing  (see  Table  1)  and  their  average  educational  level  was  14.5  (SD  = 2.4)  years 
of  formal  education.  The  two  groups  of  six  men  and  six  women  did  not  differ 
significantly  with  respect  to  age  or  education. 
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Table  1.  Information  concerning  subject  educational  background,  age, 
psychological  status,  and  hand  use. 


Sex 

Age 

Ed.a 

B&Nb 

Waterloo0 

M 

24 

16 

23 

68 

0 

M 

20 

16 

21 

67 

0 

F 

24 

18 

22 

72 

3 

F 

19 

14 

24 

65 

0 

M 

21 

16 

22 

65 

2 

F 

18 

12 

21 

64 

1 

F 

27 

16 

20 

67 

3 

F 

21 

12 

21 

66 

0 

F 

32 

18 

24 

69 

0 

M 

31 

16 

24 

65 

0 

M 

42 

18 

24 

72 

0 

M 

28 

16 

23 

60 

0 

a Educational  level 

b Briggs  & Nebes  Handedness  Inventory 
c Waterloo  Handedness  Questionnaire  - Revised 
d Beck  Depression  Inventory 
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All  12  subjects  were  strongly  right  handed  as  supported  by  their  performance 
on  the  Waterloo  Handedness  Questionnaire  (Steenhuis  and  Bryden,  1989)  which 
revealed  a mean  of  67  (SD  = 3.3)  out  of  a possible  72  representing  a strong  right  hand 
bias.  Subjects  also  demonstrated  a mean  of  22  (SD  = 1.4)  out  of  a possible  24  on  the 
Briggs  & Nebes  (1987)  handedness  inventory,  which  also  placed  them  within  a 
strongly  right  handed  designation. 

Control  Measures 

Control  Measure  1:  Prosody  Production  Within  and  Outside  of  Scanner 

This  control  measure  was  designed  to  rule  out  the  possibility  that  subjects 
were  simply  distracted  by  the  loud,  confining  environment  of  the  scanner,  which  may 
have  induced  anxiety  thereby  limiting  the  extent  to  which  they  could  have  felt  and 
sounded  happy.  The  judges  assessed  the  prosodic  accuracy  of  the  subject’s  responses 
during  experimental  testing  within  the  MR  scanner  environment  (+2  accurate,  +1 
closely  matched  target,  0 no  match  to  target  prosody).  The  averages  for  each 
condition  (happy,  angry  and  neutral)  for  each  individual  subject  were  calculated  and  a 
prosodically  correct  percentage  was  calculated. 

Additionally,  Control  Measure  1 was  performed  to  ensure  that  even  if  the  loud 
acoustic  background  noise  of  the  scanner  precluded  recording  the  subject’s  voice  in 
the  scanner,  the  subject’s  renditions  of  emotional  prosody  could  be  evaluated  from 
recordings  made  outside  the  scanner  environment  that  same  day. 

As  noted  in  Table  2 it  was  not  possible  to  assess  the  in-scanner  verbal 
production  for  subjects  sOl  and  s02  due  to  an  unexpected  technical  problem.  It  was 
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observed  that  when  audio-recording  was  attempted  from  a tape  recorder  that  was 
plugged  into  a electric  wall  receptacle  for  power,  the  recording  was  unintelligible 
because  of  a RF  carrier  noise  offset.  In  essence,  the  microphone  cord  acted  as  a 
antenna  and  re-broadcast  a harmonic  of  the  original  RF  pulse  thereby  disrupting  the 
audio-recording.  Proper  grounding  of  the  electric  receptacles  in  the  MR  console 
room  would  have  eliminated  this  carrier  noise.  This  problem  was  eliminated  in  later 
experimental  trials  by  operating  the  tape  recorder  from  battery  power.  Overall  the  use 
of  a bi-directional  microphone  appeared  to  be  effective  and  it  partially  cancelled  the 
very  loud  background  noise  produced  by  the  MRI  scanner.  The  judges  noted  that 
while  a number  of  subjects  utilized  a low  speaking  volume  during  the  experimental 
trials,  it  was  still  possible  to  hear  their  production  on  the  audiotape,  above  the 
background  noise  of  the  scanner. 

The  prosodic  accuracy  scores  for  the  12  subjects  in  the  study  are  depicted  in 
Table  2 with  group  comparisons  depicted  in  Table  3.  Almost  without  exception, 
subjects  produced  neutral  prosody  with  100%  accuracy  both  within  and  outside  of  the 
scanner  environment.  Additionally,  the  overall  average  for  all  12  subjects  revealed 
comparable  levels  of  prosodic  accuracy  for  both  emotional  prosodies  inside  and 
outside  the  MRI  scanner  environment.  For  instance,  during  the  production  of  happy 
prosody  the  overall  average  for  all  12  subjects  was  80%  inside,  compared  to  75% 
outside.  During  the  production  of  angry  prosody  inside  the  scanner  the  overall 
average  for  all  12  subjects  was  rated  to  be  62%  prosodically  accurate,  while  outside  it 
was  rated  to  be  68%  accurate.  These  figures  also  reveal  the  differential  accuracy  for 
each  emotional  prosody,  with  higher  accuracy  ratings  for  the  production  of  happy 
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prosody,  than  for  angry  prosody.  The  grouped  comparisons  depicted  in  Table  3 (2 
tailed  t-test,  p < 0.05)  revealed  no  significant  difference  between  in-scanner  vs.  out  of 
scanner  prosodic  accuracy  within  each  of  the  three  emotional  prosodies  (e.g.,  Angry- 
In  vs.  Angry-Out).  However,  there  were  significant  differences  (see  Table  3)  when 
the  scanner  condition  (e.g.,  In  or  Out)  was  held  constant  and  comparisons  were  made 
across  each  of  the  three  emotional  prosodies.  The  only  exception  to  this  was  the 
Angry-Out  vs.  Happy-Out  comparison  revealed  a trend,  but  did  not  reach  significance 
at  p < .06. 

These  findings  support  the  hypothesis  that  subjects  were  not  distracted  by  the 
loud,  potentially  anxiety  provoking  environment  of  the  scanner,  and  produced  a 
comparable  level  of  prosodic  accuracy  whether  they  were  within  or  outside  the 
scanner.  The  agreement  between  the  two  judges  for  this  control  measure  across  the 
available  10  subjects  was  greater  than  96%  for  the  trials  within,  as  well  as  outside  the 
scanner. 

Control  Measure  2:  Subjective  Evaluation  of  Emotional  Experience 

This  control  measure  was  designed  to  assess  the  possibility  that  subjects  had 
actually  experienced  an  emotion  other  than  the  target  emotion  during  the 
experimental  trials,  even  when  their  prosodic  production  matched  the  target  prosody. 
Therefore  Control  Measure  2 provided  subjects  with  a questionnaire  which  was  filled 
out  after  the  scan  to  probe  their  subjective  perception  of  their  prosodic  production. 
They  were  asked  to  write  answers  to  questions  such  as  “When  you  were  saying  the 
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sentence  with  happy  prosody,  did  you  actually  feel  happy,  or  did  your  voice  only 
sound  happy?”. 

The  subjects’  responses  to  this  post-scan  subjective  evaluation  of  emotional 
experience  revealed  a presumed  match  of  target  prosody  to  emotional  experience  of 
100%  for  neutral  and  happy  trials  (12  out  of  12  subjects).  Eleven  out  of  the  12 
subjects  reported  that  during  their  production  of  angry  prosody  they  actually  felt 
angry,  while  one  subject  reported  that  they  did  not  feel  angry  during  the  production  of 
angry  prosody.  Thus  in  general  the  subjects  stated  that  their  emotional  state  at  the 
time  of  the  verbal  prosodic  production  in  the  scanner  matched  the  target  emotional 
prosody  they  were  producing. 

Experimental  Comparisons 

A small  artifact  was  noted  to  occur  in  the  images  due  to  a radio  frequency 
carrier  noise  offset  or  a DC  signal  artifact.  This  was  likely  caused  by  the  presence  of 
the  ungrounded  microphone  cord  in  the  MR  scanning  room.  During  experimental 
trials  the  scanner  door  was  left  slightly  ajar  to  permit  the  microphone  cord  to  pass 
through.  This  also  could  have  contributed  to  this  artifact.  This  “zipper”  (Kruggel, 
von  Cramon,  & Descombes,  1999;  Haacke,  Brown,  Thompson,  & Venkatesan,  1999) 
artifact  occurred  across  phase  encoding  and  did  not  appear  to  induce  distortions  of  the 


datasets. 
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Initial  Processing 

A prosody  vs.  null  hypothesis  Student  t-test  comparison  was  performed  voxel- 
by-voxel  for  each  of  the  three  conditions.  A very  stringent  statistical  threshold  (p  < 
0.000001)  for  positive  voxels  was  utilized  because  of  the  inherently  high  statistical 
power  of  these  initial  gross  comparisons.  Large  volumes  of  significant  clusters 
(>500  pL)  were  revealed  bilaterally  for  each  of  the  three  comparisons  (see  Figures  7- 
10).  For  instance,  the  happy  prosody  vs.  null  comparison  (see  Table  4 and  Figure  8) 
revealed  a large  volume  of  activity  in  the  right  precentral  gyrus  and  precuneus  region, 
with  a smaller  volume  of  activity  noted  to  occur  in  the  left  precentral  gyrus  and 
insula,  as  well  as  the  right  superior  temporal  and  postcentral  gyrus,  and  the  right 
caudate  region.  The  angry  prosody  vs.  null  hypothesis  comparison  (see  Table  5 and 
Figure  9)  revealed  a similar  pattern  of  activity  with  a large  volume  in  the  precentral 
gyrus  bilaterally  (R  > L),  with  lesser  amounts  in  the  right  superior  frontal,  inferior 
parietal,  precuneus  and  postcentral  gyrus.  The  neutral  prosody  vs.  null  hypothesis 
comparison  (see  Table  6 and  Figure  7)  revealed  a large  volume  of  activity  in  the  right 
precentral  gyrus,  and  lesser  amounts  in  the  left  precentral  gyrus,  right  superior  frontal 
gyrus.  It  is  noteworthy  that  while  right  precuneus  activity  was  noted  to  occur 
medially  in  the  two  emotionally  related  comparisons  (angry  and  happy),  this  was  not 
noted  to  occur  in  the  neutral  prosody  vs.  null  hypothesis  comparison  (see  Figure  10). 

To  provide  more  specific  experimental  comparisons,  voxel-by-voxel  Student 
t-test  (p  < 0.001)  contrasts  were  performed  between  the  functional  MR  datasets 
related  to  the  expression  of  emotional  prosody  and  those  created  during  the 
expression  of  neutral  prosody  for  all  subjects.  In  contrast  to  the  initial  prosody  vs. 
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null  hypothesis  comparisons,  this  technique  was  utilized  because  it  had  the  potential 
of  reducing  artifactual  brain  activity  related  to  head  movement  by  eliminating  from 
analysis  the  initial  4.41s  during  which  the  stimulus  was  presented  and  speech  was 
produced,  thereby  more  clearly  isolating  task  related  activity. 


A B C 


Figure  7.  Significant  (p  < 0.00001)  activity  was  noted  to  occur  bilaterally  as  a result 
of  a neutral  prosody  vs.  null  hypothesis  comparison  averaged  across  all  12  subjects. 
These  large  significant  clusters  (>  500  pL)  were  displayed  in  images  from  the  left 
para-sagittal  plane  (A),  a matching  axial  (B)  image,  and  a para-sagittal  image  (C) 
from  the  right  hemisphere. 


Figure  8.  Significant  (p  < 0.00001)  activity  was  noted  to  occur  bilaterally  as  a result 
of  a happy  prosody  vs.  null  hypothesis  comparison  averaged  across  all  12  subjects. 
These  large  significant  clusters  (>  500  pL)  were  displayed  in  images  from  the  left 
para-sagittal  plane  (A),  a axial  (B)  image,  with  a matching  para-sagittal  image  (C) 
from  the  right  hemisphere. 
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Figure  9.  Significant  (p  < 0.00001)  activity  was  noted  to  occur  bilaterally  as  a result 
of  a angry  prosody  vs.  null  hypothesis  comparison  averaged  across  all  12  subjects. 
These  large  significant  clusters  (>  500  pL)  were  displayed  in  images  from  the  left 
para-sagittal  plane  (A),  a axial  (B)  image,  with  a matching  para-sagittal  image  (C) 
from  the  right  hemisphere. 


Figure  10.  Significant  (p  < 0.00001)  activity  was  noted  to  occur  medially  as  a result 
of  the  prosody  vs.  null  hypothesis  comparisons  averaged  across  all  12  subjects. 

These  significant  clusters  (>  500  pL)  were  displayed  in  the  mid-sagittal  plane  for  the 
following  comparisons:  (A)  neutral  prosody  vs.  null  hypothesis,  the  (B)  happy 
prosody  vs.  null  hypothesis,  and  the  (C)  angry  prosody  vs.  null  hypothesis. 
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Experimental  Comparison  1:  Happy  vs.  Neutral  Prosody 

As  depicted  in  Table  7,  the  significant  brain  activity  based  on  averaged  data 
for  all  12  subjects  demonstrated  only  a single  volume  of  activity  in  the  right 
hemisphere.  It  is  noteworthy  that  no  activity  was  revealed  in  the  traditionally  defined 
language  cortices  of  the  left  hemisphere,  even  though  this  was  a language  production 
task.  This  area  of  activity  were  limited  in  volume  and  was  confined  to  the  precentral 
gyrus  of  the  right  hemisphere  (see  Figure  11). 


Figure  1 1.  Significant  (p  < 0.001)  activity  was  noted  to  occur  in  the  right  precentral 
gyrus  as  a result  of  a t-test  comparison  between  the  production  of  happy  vs.  neutral 
prosody  averaged  across  all  12  subjects.  No  activity  was  present  in  the  para-sagittal 
image  (A)  from  the  left  hemisphere,  but  a significant  cluster  (>  100  pL)  was  present 
in  the  right  hemisphere  as  depicted  in  the  axial  plane  (B),  and  the  matching  para- 
sagittal image  (C). 


A 


B 


C 
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Experimental  Comparison  2:  Angry  vs.  Neutral  Prosody 

As  depicted  in  Table  8,  the  significant  brain  activity  averaged  across  all  12 
subjects  demonstrated  exclusive  engagement  of  the  right  hemisphere  without  any  left 
hemisphere  engagement.  The  strongest  areas  of  activity  were  in  the  right  superior 
and  middle  frontal  gyrus  (see  Figures  12  and  13),  with  additional  areas  of  activity  in 
the  right  posterior  cingulate  (see  Figure  14),  and  the  right  angular  gyrus  (see  Figure 
15).  Again  it  is  noteworthy  that  no  activity  was  revealed  in  left  hemisphere,  even 
though  this  was  a language  production  task. 


Figure  12.  Significant  (p  < 0.001)  activity  was  noted  to  occur  in  the  right  superior 
frontal  gyrus  as  a result  of  a t-test  comparison  which  contrasted  the  production  of 
angry  vs.  neutral  prosody  averaged  across  all  12  subjects.  This  significant  cluster  (> 
100  pL)  was  displayed  in  the  sagittal  (A)  and  matching  axial  (B)  plane. 
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Figure  13.  Significant  (p  < 0.001)  activity  was  noted  to  occur  in  the  right  middle 
frontal  gyrus  as  a result  of  a t-test  comparison  between  the  production  of  angry  vs. 
neutral  prosody  across  all  12  subjects.  This  significant  cluster  (>  100  pL)  was 
displayed  in  the  sagittal  (A)  and  matching  axial  plane  (B). 


A B 


Figure  14.  Significant  (p  < 0.001)  activity  was  noted  to  occur  in  the  right  posterior 
cingulate  cortex  as  a result  of  a t-test  comparison  between  the  production  of  angry  vs. 
neutral  prosody  averaged  across  all  12  subjects.  This  significant  cluster  (>  100  pL) 
was  displayed  in  the  sagittal  (A)  and  axial  (B)  planes. 
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Figure  15.  Significant  (p  < 0.001 ) activity  was  noted  to  occur  in  the  right  angular 
gyrus  as  a result  of  a t-test  comparison  between  the  production  of  angry  vs.  neutral 
prosody  averaged  across  all  12  subjects.  This  significant  cluster  (>  100  pL)  was 
displayed  in  the  (A)  axial  and  (B)  sagittal  planes. 


Experimental  Comparison  3:  Angry  vs.  Happy  Prosody  This  comparison  did  not 
reveal  any  significant  areas  of  activation. 


Experimental  Comparison  4:  Happy  vs.  Angry  Prosody  This  final  comparison 
contrasted  the  production  of  happy  prosody  contrasted  with  negative  valence  (see 
Table  8).  This  comparison  revealed  wide  ranging  activity  including  the  only  cluster 
of  left  frontal  gyrus  activity  among  all  fourof  the  comparisons.  Engagement  of  the 
left  hemisphere  was  extensive  at  a volume  of  771  pL,  but  right  hemisphere  volume 
was  still  greater  at  a volume  of  1354  pL.  The  greatest  regions  of  activity  were  noted 
to  occur  in  the  right  insula,  angular  gyrus,  and  the  right  superior  parietal  lobule.  More 
limited  volumes  of  activity  also  occurred  in  the  right  superior  frontal  gyrus  and  the 
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posterior  cingulate.  Activity  in  the  left  hemisphere  was  revealed  in  the  parietal 
precuneus.  SMA,  and  the  left  insula. 

Summary  of  Results 

Control  Measures 

Based  on  the  results  of  Control  Measure  1 , which  depicted  comparable  ratings 
both  within  and  outside  the  MRI  scanner  for  the  prosodic  accuracy  for  all  12  subjects, 
it  was  concluded  that  the  subjects  were  not  distracted  by  the  loud  environment  of  the 
scanner.  Furthermore,  this  provided  evidence  that  the  confining  setting  inside  the 
magnet  did  not  induce  anxiety,  which  could  have  limited  the  extent  to  which  subjects 
expressed  the  target  emotional  prosodies.  Furthermore,  the  results  of  the  subjective 
questionnaires  which  subjects  completed  as  part  of  Control  Measure  2 provided 
support  for  the  hypothesis  that  during  the  experimental  trials  the  subjects  may  have 
actually  experienced  the  target  emotion  associated  with  the  cued  prosody. 

Research  Question  1 

The  initial  gross  prosody  vs.  null  hypothesis  analysis  revealed  large  bilateral 
(R  > L)  areas  of  activity  across  all  three  comparisons,  most  prominently  in 
motorically  related  cortices  such  as  the  prefrontal,  and  superior  frontal  gyri  (see  Table 
10).  It  is  noteworthy  that  Table  10  did  reveal  extensive  differences  in  the  volumes  of 
the  prosody  vs.  null  hypothesis  comparisons.  For  example,  the  happy  vs.  null 
hypothesis  comparison  was  approximately  25%  larger  in  volume  than  the  angry  vs. 
null  hypothesis,  and  was  approximately  50%  larger  than  the  neutral  vs.  null 
hypothesis  comparison.  This  variability  may  have  induced  a degree  of  error  in  the 
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comparisons.  For  instance,  this  large  difference  in  volume  between  angry  or  happy, 
may  have  skewed  the  results  when  each  was  compared  to  the  fairly  limited  volumes 
associated  with  neutral  prosody. 

Because  Experimental  Comparisons  1 & 2 did  not  reveal  activity  in  known 
language  cortices  such  as  the  left  inferior  gyrus,  this  suggests  that  this  pseudo-event 
related  technique  effectively  minimized  the  fMRI  activity  related  to  the  common 
language  and  motoric  aspects  of  speech  production.  Table  6 demonstrates  that  only 
the  right  precentral  region  was  activated  as  a result  of  the  happy  vs.  neutral  prosody 
comparison.  While  the  volume  of  this  activated  region  was  limited,  this  motorically 
related  region  did  reach  the  stringent  statistical  threshold  set  for  this  comparison  (see 
Figure  11).  Finally,  Table  7 depicts  the  angry  vs.  neutral  prosody  results,  which 
revealed  exclusive  right  hemisphere  activity  (see  Figure  12-13)  as  a result  of  this 
comparison. 

While  it  is  problematic  to  evaluate  the  findings  of  the  final  two  comparisons 
(Experimental  Comparisons  3 & 4),  they  too  revealed  greater  right  than  left 
hemisphere  activity.  Each  of  these  depict  the  brain  regions  that  were  required  to 
produce  a particular  emotional  prosody,  over  and  above  the  activity  required  to 
produce  the  other  emotional  prosody.  For  instance,  no  significant  activity  was 
revealed  for  the  angry  vs.  happy  prosody  comparison.  Yet,  the  results  of  the  happy 
vs.  angry  prosody  comparison  (Table  8),  reveals  strong  right  hemisphere  activity, 
with  only  limited  left  hemisphere  activity. 
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Research  Question  2 

As  depicted  in  Tables  4-6  all  three  gross  comparisons  of  overt  speech 
production  vs.  null  hypothesis  induced  greater  right  than  left  hemisphere  activity  with 
the  largest  clusters  located  in  the  right  frontal  region  (Table  10).  In  both  the  happy 
vs.  neutral  (Table  8)  and  the  angry  vs.  neutral  prosody  (Table  7)  comparisons  the 
largest  clusters  were  again  located  in  the  right  frontal  region  (Table  10). 
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Table  2.  In-scanner  vs.  out-of-scanner  prosodic  accuracy  as  determined  by  two  raters 
for  all  12  subjects. 


Emotional  Prosody 


Happy 

Ina 

Outb 

Angry 

Ina 

Outb 

Neutral 

Ina 

Outb 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

NAC 

32/64% 

8/67% 

30/60% 

7.5/63% 

50/100% 

12/100% 

30/60% 

7/58% 

27/54% 

4.5/38% 

50/100% 

12/100% 

50/100% 

12/100% 

48/96% 

11.5/96% 

50/100% 

12/100% 

33.5/67% 

9.5/79% 

41/82% 

10.5/88% 

50/100% 

11.5/96% 

40.5/81% 

9/75% 

18.5/37% 

6.5/54% 

50/100% 

12/100% 

38/76% 

8/67% 

26/52% 

7.5/63% 

50/100% 

12/100% 

44/88% 

10/83% 

27/54% 

9/75% 

50/100% 

12/100% 

48.5/97% 

11.5/96% 

45/90% 

10.5/88% 

50/100% 

12/100% 

49/98% 

5/83% 

20/40% 

5.5/46% 

50/100% 

12/100% 

37/74% 

9.5/79% 

27/54% 

9/75% 

50/100% 

12/100% 

40/80% 

8.95/75% 

31/62% 

8.2/68% 

50/100% 

11.95/99% 

7.4 

2.1 

10.2 

2.3 

0.0 

0.2 

a In-scanner  trials  composed  of  25  trials  for  each  emotional  prosody.  Percentage  is 
based  on  25  trials  with  a maximum  possible  score  of  50  (2  = accurate  prosody,  1 = 
close  to  target  prosody,  0 = incorrect  prosody). 

b Out-of-scanner  trials  were  composed  of  6 trials  for  each  emotional  prosody. 
Percentage  is  based  on  6 trials  with  a maximum  possible  score  of  12  (2  = accurate 
prosody,  1 = close  to  target  prosody,  0 = incorrect  prosody). 
c Due  to  a technical  problem  the  in-scanner  prosody  was  not  able  to  be  recorded. 
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Table  3.  T-test  comparisons  (p  < 0.05)  between  in-scanner  vs.  out-of-scanner 
prosodic  accuracy  as  determined  by  two  raters  for  all  12  subjects. 


Comparisons  Significance  Level 


Angry-In  vs.  Angry-Out 

.098 

Happy-In  vs.  Happy-Out 

.472 

Neutral-In  vs.  Neutral-Out 

.343 

Angry-In  vs.  Happy-In 

.025* 

Angry-In  vs.  Neutral-In 

.001* 

Happy-In  vs.  Neutral-In 

.002* 

Angry-Out  vs.  Neutral-Out 

.001* 

Happy-Out  vs.  Neutral-Out 

.001* 

Angry-Out  vs.  Happy-Out 

.062 

* p < 0.05 
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Table  4.  Volume  (|iL)  of  significant  (p  < 0.00001)  fMRI  clusters  (>  500  pL)  as  a 
result  of  a happy  prosody  vs.  null  hypothesis  t-test  comparison  averaged  across  all  12 
subjects. 


BAa 

Region 

Talairach  Coordinates 

Volume/pL 

a: 

y 

z 

6 

R precentral  gyrus 

51.0 

-1.0 

40.0 

15158 

6 

R precentral  gyrus/SMAb 

0.0 

6.0 

54.0 

11373 

7 

R parietal  precuneus 

13.0 

-68.0 

34.0 

4114 

9 

R precentral  gyrus 

-44.0 

-18.0 

39.0 

4043 

13 

L insula 

-48.0 

-29.0 

20.0 

1686 

6 

L precentral  gyrus 

-53.0 

-6.0 

9.0 

1680 

R caudate/white  matter 

6.0 

7.0 

14.0 

1174 

7 

R postcentral  gyrus 

10.0 

-46.0 

61.0 

999 

42 

R superior  temporal  gyrus 

57.0 

-31.0 

12.0 

704 

R caudate/white  matter 

8.0 

19.0 

10.0 

646 

4 

R precentral  gyrus 

35.0 

-27.0 

55.0 

577 

a BA  = Brodmann’s  area 
b SMA  = supplemental  motor  area 
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Table  5.  Volume  (pL)  of  significant  (p  < 0.00001)  fMRI  clusters  (>  500  pL)  as  a 
result  of  a angry  prosody  vs.  null  hypothesis  t-test  comparison  averaged  across  all  12 
subjects. 


BAa 

Region 

Talairach  Coordinates 

Volume/pL 

X 

y 

z 

6 

R precentral  gyrus 

53.0 

-2.0 

38.0 

12531 

4 

L precentral  gyrus 

-43.0 

-17.0 

37.0 

9026 

6 

R superior  frontal  gyrus 

2.0 

6.0 

54.0 

6607 

40 

R inferior  parietal  lobule 

62.0 

-29.0 

26.0 

1343 

7 

R parietal  precuneus 

19.0 

-67.0 

48.0 

1328 

7 

R postcentral  gyrus 

28.0 

-23.0 

61.0 

549 

7 

R postcentral  gyrus 

10.0 

-48.0 

61.0 

530 

a BA  = Brodmann’s  area 
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Table  6.  Volume  (pL)  of  significant  (p  < 0.00001)  fMRI  clusters  (>  500  pL)  as  a 
result  of  a neutral  prosody  vs.  null  hypothesis  t-test  comparison  averaged  across  all  12 
subjects. 


BAa 

Region 

Talairach  Coordinates 

Volume/pL 

X 

y 

z 

6 

R precentral  gyrus 

52.0 

-3.0 

26.0 

10866 

4 

L precentral  gyrus 

-48.0 

-16.0 

44.0 

5620 

6 

R superior  frontal  gyrus/SMAb  2.0 

6.0 

53.0 

5187 

8 

R superior  frontal  gyrus 

30.0 

43.0 

39.0 

662 

a BA  = Brodmann’s  area 
b SMA  = supplemental  motor  area 
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Table  7.  Experimental  Comparison  1 : volume  (pL)  of  significant  (p  < 0.001)  fMRI 
clusters  (>  1 00  pL)  as  a result  of  a happy  vs.  neutral  prosody  t-test  comparison 
averaged  across  all  12  subjects. 


BAa  Region 

Talairach  Coordinates  Volume/pL 

4 R precentral  gyrus 

x y z 

61.0  0.0  20.0  173 

a BA  = Brodmann’s  area 
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Table  8.  Experimental  Comparison  2:  volume  (pL)  of  significant  (p  < 0.001)  fMRI 
clusters  (>  100  pL)  as  a result  of  a angry  vs.  neutral  prosody  t-test  comparison 
averaged  across  all  12  subjects. 


BAa 

Region 

Talairach  Coordinates 

Volume/pL 

X 

y 

z 

9 

R superior  frontal  gyrus 

6.0 

54.0 

27.0 

283 

9 

R middle  frontal  gyrus 

37.0 

32.0 

35.0 

151 

31 

R posterior  cingulate 

1.0 

-28.0 

41.0 

121 

39 

R angular  gyrus 

42.0 

-43.0 

24.0 

117 

a BA  = Brodmann’s  area 
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Table  9.  Experimental  Comparison  4:  volume  (pL)  of  significant  (p  < 0.001)  fMRI 
clusters  (>  100  pL)  as  a result  of  a happy  vs.  angry  prosody  t-test  comparison 
averaged  across  all  12  subjects. 


BAa 

Region 

Talairach  Coordinates 

Volume/pL 

X 

y 

z 

13 

R insula 

46.0 

-14.0 

11.0 

397 

39 

R angular  gyrus 

-30.0 

-38.0 

36.0 

365 

7 

R superior  parietal  lobule 

27.0 

-55.0 

43.0 

269 

7 

L parietal  precuneus 

-5.0 

-61.0 

45.0 

225 

7 

L parietal  precuneus 

-16.0 

-60.0 

45.0 

217 

6 

R superior  frontal  gyrus 

16.0 

18.0 

52.0 

210 

6 

L SMAb 

-4.0 

8.0 

47.0 

177 

13 

L insula 

-46.0 

-35.0 

22.0 

152 

31 

R posterior  cingulate 

1.0 

-27.0 

42.0 

113 

a BA  = Brodmann’s  area 
b SMA  = supplemental  motor  area 
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Table  10.  Overall  volume  (pL)  of  hemispheric  activity  associated  with  all 
comparisons,  averaged  across  all  12  subjects. 


Comparison  Hemisphere  Frontal  Lobe 


Left 

Right 

Left 

Right 

Happy  vs.  Null  Hypothesis 

3,366 

38,788 

1,680 

31,151 

Angry  vs.  Null  Hypothesis 

9,026 

22,888 

9,026 

19,138 

Neutral  vs.  Null  Hypothesis 

5,620 

16,715 

5,620 

16,715 

Happy  vs.  Neutral  Prosody 

0 

173 

0 

173 

Angry  vs.  Neutral  Prosody 

0 

672 

0 

434 

Happy  vs.  Angry  Prosody 

771 

1,354 

329 

607 

Angry  vs.  Happy  Prosody 

0 

0 

0 

0 
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Figure  16.  In-scanner  vs.  out-of-scanner  prosodic  accuracy  as  determined  by  two 
raters  for  all  12  subjects. 


CHAPTER  4 
DISCUSSION 

Introduction 

The  purpose  of  this  study  was  to  test  whether  the  production  of  emotional 
prosody  was  lateralized.  The  right  hemisphere  dominance  hypothesis  posited  that 
greater  brain  activity  would  be  present  in  the  right  hemisphere  regardless  of  the 
valence  of  the  emotional  prosody.  Alternatively  the  valence  hypothesis  posited  that 
the  brain  activity  would  be  greater  in  the  left  hemisphere  during  the  production  of 
positive  prosody  and  greater  in  the  right  during  the  production  of  negative  prosody. 
These  competing  hypotheses  were  evaluated  with  fMRI  which  provided  the 
localization  and  volume  of  brain  activity  related  to  the  production  of  a neutral 
declarative  sentence  (e.g.,  “They  went  to  the  store”)  intoned  with  either  happy,  angry, 
or  neutral  prosody. 


Research  Question  1 

Does  the  experimental  evidence  support  the  validity  of  the  right  hemisphere 
dominance  hypothesis,  the  valence  hypothesis,  or  a third  alternative? 
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A priori  Prediction 

It  was  predicted  that  this  fMRI  experiment  would  provide  greater  support  for 
the  right  hemisphere  dominance  hypothesis  when  the  production  of  emotional 
prosody  was  compared  to  neutral  prosody  production. 

The  findings  of  the  happy  vs.  neutral  prosody  comparison  revealed  only  a 
single  area  of  activity  in  the  right  premotor  area,  while  the  angry  vs.  neutral 
comparison  revealed  significant  activity  in  the  right  middle  frontal  gyrus  and 
prefrontal  regions,  the  right  posterior  cingulate,  and  the  right  angular  gyrus.  A 
number  of  issues  should  be  resolved  prior  to  addressing  Research  Question  1 . For 
instance,  why  was  there  a difference  in  the  perceptual  ratings  of  the  prosodic  accuracy 
of  the  subjects  between  the  production  of  angry  and  happy  utterances?  Was  there  a 
fundamental  difference  in  the  acoustic  profile  between  the  production  of  happy  or 
angry  prosody?  Was  it  appropriate  to  use  neutral  prosody  as  a baseline  for  either 
angry  or  happy  prosody?  Is  there  a fundamental  difference  in  the  degree  to  which  the 
expression  of  either  happy  or  angry  prosody  requires  an  internal  emotional  response 
or  experience?  Did  the  subjects  actually  experience  happiness  or  anger  during  task 
performance?  Finally,  why  is  the  happy  vs.  neutral  prosody  comparison  so  limited  in 
volume,  in  reference  to  the  angry  vs.  neutral  contrast? 

Why  the  Difference  in  the  Ratings  of  Angry  and  Happy  Prosody? 

The  accuracy  of  the  in-scanner  happy  prosodic  utterances  (Table  16)  were 
judged  to  be  more  accurate  (80%  accuracy),  than  the  angry  prosodic  productions 
(62%  accuracy).  Pell  (1999b)  reported  strikingly  similar  findings  when  judges 
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assessed  the  prosody  productions  of  normal  controls  which  revealed  that  they  were 
more  proficient  in  transmitting  happy  prosody  (78%  accuracy)  than  they  were  in 
communicating  angry  prosody  (64%  accuracy).  Therefore,  the  difference  in  the 
ratings  of  happy  and  angry  prosody  appears  to  be  a robust  and  reliable  measurement. 
However,  it  is  not  known  why  a difference  between  these  two  prosodic  utterances 
would  exist. 

Differences  in  the  Acoustic  Profiles  of  Angry  and  Happy  Prosody? 

This  was  not  quantified  in  this  present  study.  However,  if  a different  acoustic 
profile  was  associated  with  each  emotional  prosody,  this  could  account  for  the 
mismatch  in  the  perceptual  ratings  between  angry  and  happy  prosody.  One  prosody 
may  be  easier  to  identify  than  another.  Pell  (1999a)  demonstrated  that  the  rate  of 
production  of  happy  prosody  (4.68  words  per  second)  was  more  similar  to  the  rate  for 
neutral  (4.97  words  per  second)  prosody,  than  it  was  to  the  production  of  angry 
prosody  (4.01  words  per  second).  In  a study  of  the  acoustic  parameters  of  emotional 
prosody  Pell  (1999b)  revealed  that  normal  control  subjects  produce  happy  utterances 
with  an  elevated  mean  fundamental  frequency  (Fo)  with  a significant  rise  in  Fo  when 
marking  focus  on  sentence-final  items.  Neutral  utterances  were  characterized  by  a 
less  intense  version  of  this  pattern  with  a relatively  low  mean  F0  and  an  intermediate 
amount  of  Fo  variation.  In  contrast,  the  acoustic  pattern  associated  with  the 
production  of  angry  prosody  demonstrated  a more  uniform  Fo  level.  Pell  (1999b) 
noted  a significantly  higher  mean  F0  when  happy  was  compared  to  angry  (final 
focus).  Therefore,  happy  prosody  is  acoustically  more  similar  to  neutral  prosody, 
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then  it  is  to  angry  prosody.  It  is  unknown  if  these  acoustic  similarities  would  be 
related  with  a correspondingly  similar  pattern  of  brain  activity.  If  these  brain  patterns 
were  similar,  this  could  have  minimized  the  significant  areas  of  activity  that  would 
have  been  revealed  when  happy  was  compared  to  neutral  prosody,  consistent  with  the 
findings  of  this  dissertation. 

Was  it  Appropriate  to  Utilize  Neutral  as  a Baseline  for  Emotional  Prosody? 

An  unexamined  a priori  assumption  was  that  neutral  prosody  had  a minimal 
amount  of  intonation  and  was  a suitable  measure  to  use  as  a baseline  to  examine  the 
activity  associated  with  emotional  prosodies.  However,  the  neutral  vs.  null 
hypothesis  comparison  revealed  an  extensive  degree  of  left  and  right  hemisphere 
activity.  This  brain  activity  is  likely  related  to  the  respective  linguistic  and  prosodic 
aspects  of  the  task.  Additionally,  Pell’s  (1999b)  acoustic  study  of  normal  controls 
revealed  an  intermediate  level  of  Fo  variability  during  neutral  prosody  production,  not 
a total  lack  of  variability.  As  noted  earlier,  many  have  argued  that  neutral  prosody 
operates  as  a functional  baseline  along  a prosodic  continuum  which  is  modified 
through  subtle  or  more  robust  inflections  to  express  various  emotions  (Grandour, 
Larsen,  Dechongkit,  Ponglorpisit,  & Khunadorn,  1995;  Ladd,  1980;  Liberman,  1967). 
Yet,  while  neutral  prosody  can  be  thought  of  as  a functional  baseline  for  emotional 
prosodies,  at  the  level  of  brain  activity  it  still  requires  substantial  cortical  resources  to 
support  this  function.  It  would  appear  then  that  neutral  prosody  is  a sub-optimal 
baseline  for  comparisons  of  the  brain  activity  associated  with  either  happy  or  angry 
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prosody.  A more  accurate  baseline  would  have  been  established  by  the  utilization  of 
a utterance  which  lacked  speech  prosody  (i.e.,  monotonic  or  robotic-like  voice). 

Differences  in  the  Emotional  Experience  Associated  with  Happy  or  Angry? 

It  is  known  that  it  is  more  socially  acceptable  to  display  happy  rather  than 
angry  emotions.  Therefore,  the  production  of  happy  prosody  may  be  more  practiced 
and  as  a result  could  be  more  automatically  generated.  For  example,  to  produce  a 
believable  rendition  of  happy  prosody  might  simply  require  the  utilization  of  a 
distinct  pattern  of  pitch  alterations  such  as  the  rise  in  Fo  on  sentence-final  items  that 
Pell  (1999b)  revealed,  possibly  without  an  internal  experience  of  happiness.  In 
contrast,  projecting  a credible  angry  tone  of  voice  might  require  more  subtle 
adjustments  to  vocal  quality  which  are  possible  only  after  that  emotion  has  been 
internally  experienced.  Flowever,  it  is  difficult  to  objectively  determine  whether  this 
hypothesis  is  correct. 

Yet,  if  a difference  does  exist  at  this  level,  it  would  then  be  problematic  to 
make  direct  comparisons  between  these  two  emotional  prosodies.  Table  10  does 
reveal  extensive  differences  in  the  volumes  of  the  prosody  vs.  null  hypothesis 
comparisons.  For  example,  the  happy  vs.  null  hypothesis  comparison  was 
approximately  25%  larger  in  volume  of  activity  than  the  angry  vs.  null  hypothesis 
comparison,  and  twice  as  large  as  the  neutral  vs.  null  hypothesis  comparison.  These 
findings  suggest  that  it  was  problematic  to  attempt  a direct  comparison  between  these 
two  prosodies  because  of  these  differences  in  volume.  This  then  would  explain  the 
lack  of  any  significant  activity  as  a result  of  the  angry  vs.  happy  comparison. 
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Therefore,  these  empirical  differences  in  brain  activity  revealed  in  the  happy 
vs.  null  and  the  angry  vs.  null  hypothesis  comparisons  might  possibly  represent  a 
variable  internal  emotional  experience  associated  with  either  emotional  prosody. 

Did  the  Subjects  Actually  Feel  Happy  in  the  Scanner? 

Another  possibility  is  that  while  subjects  were  judged  to  sound  happy,  and 
they  reported  that  they  were  happy  in  the  post  scan  questionnaire,  they  might  not 
actually  have  felt  happy.  If  correct,  this  would  have  limited  their  internal  emotional 
experience  of  happiness  and  then  altered  the  brain  activity  associated  with  this  task  in 
unpredictable  ways.  No  objective  assessment  is  available  that  could  accurately 
determine  whether  the  subjects  actually  experienced  happiness  during  the 
experimental  trials.  The  scanner  environment  is  confining,  and  potentially  anxiety- 
provoking,  and  could  have  been  a strong  deterrent  to  the  subjects  experiencing  a 
happy  emotion.  If  this  conjecture  is  correct,  during  the  happy  trials  of  this  experiment 
one  could  propose  that  the  subjects  were  in  essence  attempting  to  express  an  emotion 
they  did  not  truly  feel.  Having  committed  themselves  to  this  experiment  they  may 
have  adopted  the  Meisner  Technique  (Meisner  and  Longwell,  1987)  and  acted  the 
part.  That  is,  they  applied  additional  flux  to  their  vocal  intonation  to  produce  the 
target  happy  prosody,  while  the  fMRI  results  revealed  only  engagement  of  motoric 
cortices. 

During  the  production  of  angry  prosody,  it  is  possible  that  some  of  the 
subjects  were  experiencing  a degree  of  anxiety  during  the  experimental  trials.  Anger 
and  anxiety  are  different  emotions,  but  each  are  known  to  facilitate  a fight  or  flight 


85 


response.  It  is  likely  that  anger  and  anxiety  are  more  similar  to  each  other,  than  they 
are  to  happy.  For  instance,  anger  and  anxiety  have  been  more  associated  with 
preparation  for  action,  in  contrast  to  happy  which  may  be  more  passive  and  evaluative 
(Heilman,  1994).  Adopting  Stanislawsky’s  (1936)  acting  technique  known  as  The 
Method  (see  also  Strasberg,  1987),  during  the  scan  this  fight  or  flight  arousal  might 
have  accentuated  the  brain  activity  during  the  angry  trials.  This  is  an  interesting 
speculation,  yet  the  happy  vs.  null  hypothesis  comparison  was  approximately  25% 
larger  in  volume  when  compared  to  the  angry  vs.  null  hypothesis  comparison  and 
demonstrated  robust  activity  in  both  hemispheres.  Therefore,  these  findings  argue 
against  the  possibility  that  subjects  failed  to  experience  happiness  during  those  trials. 

Why  was  the  Happy  vs.  Neutral  Comparison  so  Limited  in  Activation? 

This  is  a complex  matter,  but  an  integration  of  the  preceding  discussion  can 
help  to  address  this  issue.  It  was  demonstrated  that  there  was  a significant  difference 
between  the  ratings  of  angry  and  happy  prosody  utterances,  likely  due  to  different 
acoustical  profiles  associated  with  each  prosody.  It  was  also  concluded  that  it  was 
misleading  to  utilize  neutral  prosody  as  a baseline  measure  because  of  its  acoustic 
similarity  to  happy.  This  may  have  lead  to  a reduced  amount  of  brain  activity 
revealed  in  the  happy  vs.  neutral  comparison.  Finally,  it  appeared  that  expressing 
happy  prosody  is  likely  associated  with  a somewhat  less  intense  emotional  experience 
than  is  angry  prosody.  While  this  may  have  reduced  the  volume  of  significant 
activity  in  the  happy  vs.  neutral  comparison  to  an  extent,  it  was  suggested  that  the 
greatest  amount  of  error  was  induced  through  the  use  of  neutral  prosody  as  a baseline. 
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These  findings  do  not  invalidate  this  dissertation,  but  they  do  provide  a more  detailed 
perspective  from  which  to  finally  address  Research  Question  1 . 

Conclusion  - Research  Question  1 

Therefore,  based  on  the  findings  of  this  study,  greater  support  was  found  for 
the  right  hemisphere  dominance  hypothesis.  It  is  important  to  note  that  while  these 
findings  raise  doubt  about  the  validity  of  the  valence  hypothesis,  they  cannot  entirely 
refute  that  hypothesis.  The  crucial  comparison  for  this  evaluation  was  the  happy  vs. 
neutral  comparison  which  showed  greater  right  than  left  hemisphere  activity, 
inconsistent  with  the  valence  hypothesis  which  predicted  the  opposite  lateralization 
pattern.  Yet,  because  of  the  questions  regarding  the  appropriateness  of  the  utilization 
of  neutral  prosody  as  a baseline  and  the  paucity  of  the  activity  in  the  happy  vs.  neutral 
comparison,  this  stronger  claim  can  not  be  fully  supported. 

Yet,  if  the  right  hemisphere  dominance  hypothesis  more  accurately  represents 
the  brain  dynamics  associated  with  the  production  of  emotion,  one  must  still  address 
why  other  neuroimaging  studies  (Canli  et  al.,  1998;  Imaizumi  et  al.,  1997;  Irwin  et 
al.,  1997;  Paradiso  et  al.,  1997;  Sackeim  et  al.,  1982;  Sutton  et  al.,  1997)  have 
demonstrated  left  frontal  activity  during  experiments  which  involved  positive 
valence.  This  is  highly  speculative,  but  the  source  of  this  activity  might  be  related  to 
the  utilization  of  a specific  cognitive  strategy.  That  is,  during  task  performance  the 
subjects  might  have  been  acting  to  permit  them  to  outwardly  express  an  emotion 
which  they  did  not  truly  feel,  thereby  engaging  left  frontal  resources  in  the  process. 
As  opposed  to  The  Method  (Stanislavsky,  1936)  which  encourages  performers  to  re- 
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experience  an  emotional  experience  from  their  past,  the  more  pragmatic  Meisner 
acting  technique  (Meisner  & Longwell,  1987)  may  engage  left  frontal  networks  to 
facilitate  an  optimal  performance. 

Research  Question  2 

Is  there  any  evidence  for  the  existence  of  a prosodic  output  module  in  the  right 
frontal  lobe  and  can  a cognitive  model  of  prosody  production  be  generated  from  this 
information? 

A priori  Prediction 

It  was  predicted  that  the  results  of  this  experiment  would  reveal  activity  in  the 
right  frontal  lobe  during  the  production  of  emotional  prosody  which  would  provide 
tentative  evidence  for  this  hypothesized  prosodic  output  module.  It  was  assumed  that 
only  limited  evidence  would  be  revealed  which  would  support  a cognitive  model  of 
prosody  production. 

A number  of  issues  must  be  resolved  prior  to  addressing  Research  Question  2. 
Because  of  the  potential  confounds  associated  with  the  happy  vs.  neutral  comparison 
addressed  in  the  preceding  section,  this  discussion  will  focus  on  the  angry  vs.  neutral 
comparison.  For  instance,  this  analysis  revealed  significant  activity  in  the  right 
prefrontal  lobe,  the  right  middle  frontal  gyrus,  the  right  posterior  cingulate,  and  the 
right  angular  gyrus.  Yet,  why  would  the  right  frontal  lobe  be  involved  in  the 
production  of  emotional  prosody  at  all?  What  is  the  potential  role  of  the  right 
prefrontal  region  in  this  process?  Do  prosodic  production  and  comprehension  engage 
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the  same  brain  regions?  What  is  the  significance  of  the  right  posterior  cingulate  in 
prosodic  production?  Finally,  why  would  the  right  angular  gyrus  activate  during  this 
task  and  what  possible  role  would  it  have  in  a hypothetical  cognitive  model  of  the 
production  of  angry  prosody? 

Why  would  the  Right  Frontal  Lobe  be  involved  in  Prosody  Production? 

It  is  logical  to  presume  that  prosodic  processing  would  be  facilitated  by  the 
right  hemisphere  due  to  its  preferential  role  in  attentional  monitoring  of  the 
environment  (Heilman  and  van  Den  Abell,  1980).  Lindfield,  Wingfield,  and 
Goodglass,  1999)  demonstrated  that  emotional  prosody  boosts  comprehension  of 
lexical  elements.  It  is  possible  that  the  undulating  alterations  in  Fo,  which  partially 
represent  emotional  prosody,  might  be  used  as  an  acoustic  signal  to  maintain  the 
listeners  attention.  During  development  while  the  left  frontal  region  becomes 
organized  to  produce  the  finely  resolved  sequences  of  phonemes  utilized  in 
expressive  language,  it  is  possible  that  the  right  frontal  lobe  becomes  optimized  for 
the  coordination  of  sequential  events  which  help  facilitate  the  attentional  goals  and 
processes  of  the  right  hemisphere. 

Concerning  the  active  regions  that  have  a known  role  in  motor  function,  the 
happy  vs.  neutral  prosody  comparison  revealed  activity  in  the  right  precentral  gyrus, 
while  the  angry  vs.  neutral  comparison  showed  only  activity  in  the  right  middle 
frontal  gyrus.  Why  would  this  difference  occur?  As  noted  earlier,  there  are 
indications  that  during  speech  production  Broca’s  area  (Brodmann’s  areas  44  & 45)  is 
engaged,  but  it  does  so  in  conjunction  with  associated  premotor  and  primary  motor 
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cortex  regions  (Wildgrubber,  Ackermann,  Klose,  Kardatzki,  & Grodd,  1996).  It  is 
possible  that  the  greater  alterations  in  F0  that  are  associated  with  the  production  of 
happy,  as  opposed  to  angry  prosody,  require  more  premotor  cortical  engagement  to 
facilitate  the  changes  in  oral  and  lingual  musculature  which  support  this  process. 

One  could  propose  that  the  right  inferior  frontal  region  might  actually 
coordinate  the  on-line  integration  of  the  suprasegmental  Fo  alterations  associated  with 
emotional  prosody  into  the  acoustic  speech  stream  via  cross  callosal  connections 
(Ross,  Flarney,  deLacoste-Utamsing,  & Purdy,  1981).  Watson  and  Heilman  (1983) 
did  report  a deficit  in  repeating  affective  tone  after  callosal  disconnection  had 
occurred.  Additionally,  Speedie,  Coslett,  and  Heilman,  (1984)  studied  two  patients 
with  mixed  transcortical  aphasia  after  left  hemisphere  lesions.  They  found  that  the 
patients  were  unable  to  repeat  affective  prosody,  yet  had  intact  propositional  speech. 
This  suggests  that  the  isolated  left  perisylvian  region  responsible  for  propositional 
speech  does  not  mediate  prosody.  Speedie  et  al.  (1984)  posited  that  this  information 
was  normally  transferred  to  the  left  hemisphere  from  right  hemisphere  resources  via 
the  corpus  callosum.  Finally,  Klouda,  Robin,  Graff-Radford,  and  Cooper  (1988) 
provided  acoustical  evidence  of  a specific  deficit  in  prosody  production  following 
callosal  damage.  Klouda  and  colleagues  (1988)  described  disruptions  in  F0  contours 
in  their  patient  which  paralleled  the  acoustic  deficits  which  have  been  reported  for 
right  hemisphere  patients  (Pell,  1999b).  The  intimate  neuronal  connections  between 
the  right  frontal  region  potentially  facilitating  prosody  and  the  left  frontal  region 
orchestrating  syntactical  generation  would  support  such  a hypothesis. 
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Specifically,  why  did  Prosody  Production  activate  the  Right  Prefrontal  Region? 

It  is  possible  that  the  engagement  of  this  right  prefrontal  region  is  simply 
related  to  some  motoric  aspect  of  facilitating  the  intonational  shifts  in  Fo  during 
speech  production  discussed  earlier.  The  prefrontal  regions  do  receive  substantial 
projections  from  motorically  related  cortices.  However,  many  lines  of  inquiry 
support  the  view  that  the  engagement  of  the  prefrontal  region  during  these  tasks 
appears  to  be  related  to  higher  order  aspects  of  cognition,  rather  than  the  execution  of 
simple  motoric  sequences. 

The  finding  of  significant  activity  near  the  right  frontal  pole  during  the 
production  of  angry  emotional  prosody  could  suggest  that  emotional  prosodic 
semantics  may  be  supported  by  this  region.  To  produce  an  angry  tone  of  voice  the 
subjects  would  need  to  access  a representation  of  the  acoustic  pattern  of  this  prosody 
based  on  the  visually  presented  cue.  Both  frontal  poles  are  known  to  receive  strong 
projections  from  the  limbic  cortex  and  these  frontal  regions  have  a well  established, 
but  poorly  specified,  role  in  the  control  and  modulation  of  emotional  behavior.  Since 
a specific  experimental  task  which  attempted  to  isolate  prosodic  semantics  was  not 
utilized  in  this  experiment,  only  speculative  inference  can  be  provided  as  to  the  role 
of  semantics  in  the  system  supporting  prosody  production. 

It  is  reasonable  to  consider  the  possibility  that  the  prefrontal  regions  have 
some  involvement  with  semantic  processing,  of  which  emotion  also  plays  a role.  If 
this  speculation  is  correct,  these  emotional  semantic  representations  likely  exist  in  a 
distributed  network.  Part  of  that  distributed  network  may  encode  the  prosodic  pitch 
changes  associated  with  a particular  emotion.  For  instance,  the  representation  for 
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angry  may  include  memories  of  angry  facial  expressions,  actions,  angry  words, 
gestures,  and  even  angry  tones  of  voice.  As  noted  earlier,  this  region  may  contain 
prototypic  prosodic  echoic  representations  for  particular  emotions.  Blonder  and 
colleagues  (1991)  suggested  that  these  emotional  semantic  representations  might  be 
separate  from  their  lexical  or  modular  emotional  components.  They  provided 
evidence  that  while  these  semantic  representations  were  likely  to  be  bilateral,  they 
suggested  that  the  lexical  or  modular  components  were  likely  located  in  the  right 
hemisphere. 

Why  did  the  right,  but  not  the  left  prefrontal  cortex  activate  during  these 
tasks?  Others  have  demonstrated  that  generating  words  with  emotional  connotations 
engage  the  left  prefrontal  cortex  (Crosson  et  al.,  1999b),  yet  the  production  of 
emotional  prosody  in  this  dissertation  activated  the  right  prefrontal  cortex  during  the 
expression  of  angry  prosody.  It  is  possible  that  the  proximate  neuroanatomic  location 
and  projections  from  the  right  prefrontal  cortex  (which  is  a nexus  of  those 
representations),  and  the  right  inferior  frontal  gyrus  (which  may  facilitate  the  motoric 
instantiation),  is  the  reason  for  the  prominent  right,  not  left,  prefrontal  activity  that 
was  noted  to  occur  in  this  study. 

Do  Prosody  Production  and  Comprehension  engage  the  same  Brain  Regions? 

The  present  finding  of  preferential  engagement  of  cortices  near  the  right 
frontal  pole  as  a result  of  a emotion  vs.  neutral  prosody  production  task  compliments 
one  facet  of  the  ablation  studies  of  Ross  (1981.  1988)  who  documented  prosodic 


92 


production  deficits  after  right  frontal  lesions.  However,  Ross  (1981,  1988)  also 
argued  that  prosodic  comprehension  was  facilitated  by  right  posterior  regions. 

Ross’  (1981,  1988)  hypothesis  is  difficult  to  integrate  with  the  results  of 
George  et  al.  (1996)  who  revealed  right  frontal  activity  in  a PET  study,  during  a 
prosody  comprehension  task.  One  factor  that  may  be  important  in  reconciling  these 
differences  is  that  George  and  colleagues  (1996)  did  not  organize  their  results  by 
emotional  valence  which  may  have  minimized  the  right  posterior  activity.  PET  also 
has  a fairly  low  spatial  resolution  and  more  subtle  right  posterior  activity  may  have 
been  present  in  the  George  et  al.  (1996)  study,  but  was  below  their  statistical 
threshold. 

It  is  possible  that  Ross’s  (1981,  1988)  conjecture  of  a role  of  the  right 
posterior  region  in  prosody  comprehension  was  simply  incorrect  and  that  the  system 
supporting  prosody  in  the  right  hemisphere  does  not  parallel  the  organization  of  the 
language  system  in  the  left  hemisphere.  Since  prosody  comprehension  was  not 
evaluated  in  this  dissertation,  the  validity  of  this  facet  of  his  hypothesis  was  not 
formally  assessed.  However,  Nishitani  and  Hari  (2000)  recently  provided  evidence 
from  a magnetoencephalogram  (MEG)  experiment  with  normal  subjects  that  the  left 
Brodmann’s  area  44  is  a action  execution/observation  matching  system.  This  is  based 
on  Rizzolatti  and  Arbib  (1988)’s  earlier  conjecture  of  the  existence  of  “mirror 
neurons”  in  this  region.  They  discovered  in  the  monkey  that  these  neurons  discharge 
either  when  a particular  hand  gesture  is  performed,  or  when  that  monkey  observes 
that  particular  gesture.  Rizzolatti  and  colleagues  (1988)  posited  that  these  mirror 
neurons  provide  a mechanism  for  learning  motoric  hand  gestures.  They  also 
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suggested  that  this  system  supports  the  process  by  which  we  acquire  representational 
language.  Through  observing  and  mimicking  the  motoric  oro-facial  and  oro-linguial 
gestures  of  others,  the  discharges  of  these  mirror  neurons  may  enable  the  individual  to 
match  their  speech  productions  to  an  earlier  experienced  motoric  pattern.  If  this 
conjecture  is  correct,  then  Brodmann’s  area  44  in  the  right  hemisphere  might  also 
contain  mirror  neurons  which  discharge  when  a particular  prosody  is  produced,  or 
when  that  intonation  pattern  is  perceived.  One  could  further  speculate  that  the 
language  system  in  the  left  hemisphere  expands  in  development  posteriorially  from 
this  Broca’s  area  core  to  form  the  normative  Broca’s/Wernicke’s  continuum  in 
proportion  to  the  final  representational  complexity  of  this  very  advanced  system. 

Since  prosody  is  apparently  utilized  for  fairly  simple  representational  purposes,  this 
expansion  is  more  limited.  It  is  known  that  the  prosodic  comprehension  system 
becomes  functional  much  earlier  in  development  in  contrast  to  the  propositional 
language  system  (Bell,  Davis,  Morgan-Fisher,  & Ross,  1990),  which  may  indicate 
that  it  is  a simpler,  less  complicated  system  with  a less  extensive  cortical 
representation. 

What  is  the  Significance  of  the  Right  Posterior  Cingulate  Activity? 

It  is  interesting  to  compare  the  posterior  cingulate  activity  in  this  present  study 
(see  Figure  14)  with  that  of  a recent  fMRI  experiment  by  Maddock  and  Buonocore 
(1997).  They  reported  strong  left  posterior  cingulate  gyrus  activity  when  threat 
words  were  presented  auditorily  compared  to  neutral  words.  The  posterior  cingulate 
consists  of  Brodmann's  areas  29,  20,  23,  and  31  and  contains  neurons  that  monitor 
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eye  movements  and  respond  to  sensory  stimuli.  However.  Brodmann’s  area  23  was 
the  only  significantly  active  region  of  the  posterior  cingulate  that  was  noted  in  the 
angry  vs.  neutral  comparison  of  this  study.  Ablation  studies  suggest  that  this  region  is 
involved  in  the  modulation  of  attention,  spatial  orientation  and  memory  (Sunderland 
and  Hoesing,  1993;  Valenstein  et  al.,  1987).  Studies  have  also  shown  that  the 
posterior  cingulate  cortex  has  extensive  connections  with  the  orbitofrontal  cortex  and 
the  medial  temporal  region  (Voyt  and  Pandya,  1987).  It  is  likely  that  connections 
between  posterior  cingulate  and  parahippocampal  cortices  (Goldman  Rakic,  Selemon, 
& Schwartz,  1984;  Musil  and  Olsen,  1993)  play  a role  in  processing  episodic 
memories. 

Lindfield,  Wingfield,  and  Goodglass,  1999)  demonstrated  that  emotional 
prosody  boosts  comprehension  of  lexical  elements.  It  is  possible  that  the  undulating 
Fo  shifts  in  a speaker’s  voice  which  represent  prosody,  may  be  utilized  to  focus  the 
attention  of  the  listener.  One  could  suggest  that  prosody  helps  to  maintain  a stable 
attentional  channel  between  listener  and  speaker  which  may  also  increase  the 
memorability  of  the  communicative  interaction.  The  right  posterior  cingulate,  in 
conjunction  with  the  right  prefrontal  region,  may  work  to  bind  these  elements  into  a 
memorable  episodic  event. 

Why  would  the  Right  Angular  Gyrus  Activate  during  Prosody  Production? 

The  left  angular  gyrus  is  known  to  integrate  visual  and  somesthetic 
information  necessary  for  reading.  It  is  thought  that  the  left  angular  gyrus  is  a central 
area  involved  in  the  processing  of  orthographic  lexical  word-forms.  For  instance, 
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damage  to  this  structure  induces  alexia  with  agraphia  (Dejerine,  1891).  However,  the 
capabilities  associated  with  the  right  angular  gyrus  have  not  been  well  established. 

It  may  be  that  the  right  angular  gyrus  has  a similar  decoding/storage  function 
to  that  of  the  left  angular  gyrus,  although  with  a different  focus  reflecting  the 
dominant  capabilities  of  that  hemisphere.  For  instance,  while  the  left  hemisphere  is 
specialized  for  language  and  sequential  problem-solving,  the  right  hemisphere  is 
believed  to  be  specialized  for  attentional  monitoring  and  visuospatial  tasks 
(Gazzaniga,  2000).  Therefore,  one  possibility  is  that  the  right  angular  gyrus  provides 
a graphemic/prosodic  decoding/storage  function  and  associates  these  words  with 
associated  prosodic  representations. 

Conclusion  - Research  Question  2 

Based  on  the  preceding  discussion,  the  broad  outlines  of  a cognitive  model  of 
emotional  prosody  can  be  suggested  based  on  the  evidence  in  this  dissertation,  at  least 
with  respect  to  the  angry  prosody  trials.  As  noted  earlier,  because  of  the  potential 
confounds  associated  with  the  happy  vs.  neutral  comparison  reviewed  in  the 
preceding  section,  this  conclusion  is  limited  to  the  angry  vs.  neutral  findings. 

The  process  by  which  a subject  produced  a single  angry  utterance  could  be 
suggested  based  on  this  hypothetical  model  and  integrated  with  the  empirical 
evidence.  For  instance,  the  subject  would  first  be  presented  with  the  written  sentence 
and  a cued  emotion,  such  as  angry.  They  would  have  to  decode  the  written  sentence 
and  the  prosodic  cue  and  then  access  an  echoic  representation  for  angry  prosody. 

This  graphemic/prosodic  decoding  process  might  account  for  the  activity  in  the  right 
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angular  gyrus.  This  might  have  provided  access  to  prosodic  semantics  located  in  the 
right  prefrontal  area.  This  region  likely  contains  memory  traces  of  past  angry 
experiences,  angry  tones  of  voice  etc.  While  this  cortical  processing  occurred,  a 
memory /attentional  system  might  have  been  engaged  to  maintain  these  on-line 
representations  through  networks  associated  with  the  right  posterior  cingulate. 

Finally,  a motoric  sequence  which  represented  the  production  of  angry  prosody  would 
have  been  selected  by  engaging  the  prosodic  output  module  located  in  the  right 
middle  frontal/premotor  region.  The  module  is  hypothesized  to  contain  the  sequence 
of  muscular  adjustments  necessary  to  produce  angry  prosody  verbally.  It  is  posited 
that  this  supra-segmental  information  would  then  be  shared,  via  the  corpus  callosum, 
to  Broca’s  area  in  the  left  hemisphere.  Then  this  information  would  be  passed  down 
to  cortico-bulbar  tracts  to  facilitate  the  final  motoric  production  via  oral/lingual 
musculature.  This  would  require  the  co-creation  of  a string  of  phonemes  with  a 
coherent  syntactic  structure  (left  frontal),  which  were  overlaid  with  prosodic  nuance 
(right  frontal). 

Clinical  Implications 

While  this  study  examined  the  patterns  of  brain  activity  among  neurologically 
intact  individuals,  the  findings  of  the  angry  vs.  neutral  comparison  also  help  explain 
the  potentially  disruptive  effects  of  aprosodia  on  the  lives  of  those  who  suffer  right 
hemisphere  injury.  For  instance,  clinically  the  flat  affect  of  right  hemisphere  patients 
is  often  misinterpreted  as  a sign  of  depression  by  health  care  workers.  Knowledge  of 
the  difference  between  affective  disorders  and  aprosodia  as  a result  of  neurological 
injury  can  help  in  the  process  of  assessment. 
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These  findings  support  what  experienced  clinicians  already  know,  that  it  is 
always  appropriate  to  assess  emotional  prosody  in  patients  with  right  hemispheric 
injury  and  to  treat  these  more  subtle  communication  deficits  (for  example  see 
Stringer,  1996).  This  information  could  be  used  to  inform  families  of  potential 
mismatches  between  the  internal  emotional  experience  of  the  patient  and  their 
possibly  limited  repertoire  of  emotional  expressions  utilized  in  verbal 
communication.  This  mismatch  has  the  potential  of  hindering  the  patient’s  ability  to 
effectively  communicate  in  the  home  setting  and  provides  a handicap  in  social 
situations.  The  co-occurrence  of  a deficit  in  prosodic  comprehension  further 
aggravates  this  situation. 

Finally,  these  results  compliment  a recent  study  by  Etcoff,  Ekman,  Magee  and 
Frank  (2000)  which  demonstrated  that  left  hemisphere  patients  with  aphasia  were 
significantly  more  accurate  than  controls  at  detecting  lies  based  on  either  facial  or 
vocal  cues.  This  suggests  that  damage  to  the  networks  underlying  language 
comprehension  may  result  in  the  development  of  compensatory  skills  in  recognizing 
non-verbal  behavior,  possibly  located  in  the  intact  right  frontal  lobe. 

Future  Research 

Further  advancement  of  statistical  and  methodological  techniques  such  as 
rapid,  random  event-related  fMRI  (Burock,  Buckner,  Woldorff,  Rosen,  & Dale,  1999; 
Grabowski  & Damasio,  1996)  will  help  depict  the  complex  patterns  of  brain  activity 
associated  with  overt  speech  production.  While  the  recent  study  by  Barch  et  al., 
(1999)  validates  the  use  of  overt  speech  in  a fMRI  experimental  paradigm,  they  do 
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not  recommend  the  use  of  single  subject  analysis.  They  argue  that  group  analysis  are 
still  required  because  of  limited  statistical  power  at  the  individual  subject  level. 
However,  it  would  be  optimal  to  observe  the  patterns  of  fMRI  activity  for  single 
subjects,  especially  with  respect  to  studies  of  neurological  patients  with  aprosodic 
deficits.  The  techniques  developed  for  this  dissertation  have  helped  to  lay  the 
foundation  for  single  subject  studies  which  will  examine  those  in  vivo  patterns  in 
more  detail.  The  most  promising  future  studies  will  probe  the  patterns  of  brain 
activity  originating  from  patients  who  demonstrate  aprosodia  after  brain  lesions. 
Detailed  behavioral  assessment  of  these  aprosodic  patients  can  be  correlated  with  in 
vivo  fMRI  brain  activity  and  compared  with  normative  patterns  of  brain  activity  from 
subjects  that  have  not  sustained  neurological  injury.  Then  detailed  comparisons  can 
be  made  between  the  production  of  non-linguistic  (emotional  and  non-emotional)  and 
linguistic  prosody  to  better  characterize  these  relationships. 

Finally,  these  techniques  will  permit  the  examination  of  the  changing  patterns 
of  brain  activity  associated  with  the  natural  course  of  recovery  after  neurological 
injury.  These  techniques  potentially  will  allow  comparisons  between  the  networks 
engaged  before  and  after  treatment,  thereby  providing  a technique  to  assess  the 
efficacy  of  various  treatment  strategies. 

Summary 

The  finely  spatial  resolution  of  fMRI  has  permitted  an  extension  of  the  main 
finding  of  landmark  neurological  studies  (Tucker,  Watson,  & Heilman,  1977;  Ross  & 
Mesulam,  1979)  which  revealed  a role  of  the  right  hemisphere  in  the  production  of 
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emotional  prosody.  These  in  vivo  findings  with  normal  subjects  has  provided  further 
support  for  their  right  hemisphere  dominance  theory  and  provided  tentative  evidence 
that  a prosodic  output  module  may  be  operative  in  the  right  frontal  lobe. 


APPENDIX  A 

PRE-SCAN  SUBJECT  CHECKLIST 


1 . Name?  

2.  Age?  

3.  Date  of  Birth 

4.  Weight  (for  setting  scanner  parameters) 

5.  Email  address  (to  send  tif  image  to) 

6.  Mailing  address  (to  send  glossy  image  to) 

7.  Have  you  had  a history  of  long  lasting  (chronic)  depressions  or  psychiatric 

illness?  Any  history  of  this  in  your  immediate  family?  

8.  Have  you  ever  had  a significant  injury  to  your  head,  loss  of  consciousness, 

possibly  after  an  automobile  or  a bicycle  accident?  

9.  Any  history  of  a learning  disorder,  dyslexia,  or  a attention  deficit  disorder? 


10.  What  is  the  highest  educational  degree  you  have  obtained?  

1 1 . Is  English  the  first  language  you  learned  to  speak?  Do  you  speak  any  other 

languages  and  at  what  age  did  you  acquire  them?  

12.  What  is  your  ethnic  background  (country  of  origin  of  your  parents  or 

grandparents)?  

13.  Have  your  ever  worked  as  a professional  actor/actress,  DJ?  
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APPENDIX  B 

SUBJECT’S  POST-SCAN  QUESTIONNAIRE 


1 . When  you  were  in  the  scanner  and  asked  to  produce  a happy  tone  in  your  voice, 

do  you  think  it  actually  sounded  happy?  

2.  When  you  were  saying  the  sentence  with  happy  prosody,  did  you  actually  “feel" 
happy,  or  did  your  voice  only  sound  happy? 

3.  When  you  were  in  the  scanner  and  asked  to  produce  a neutral  tone  in  your  voice, 

do  you  think  it  actually  sounded  neutral?  If  you  think  it  sounded  emotional, 
which  emotion  was  it  closest  to?  

4.  When  you  were  saying  the  sentence  with  neutral  prosody,  did  you  actually  “feel" 
emotionally  neutral,  or  did  your  voice  only  sound  emotionally  neutral?  If  you 
didn’t  feel  emotionally  neutral,  which  emotion  were  you 

experiencing? 

5.  When  you  were  in  the  scanner  and  asked  to  produce  an  angry  tone  in  your  voice, 
do  you  think  your  voice  actually  sounded  angry? 

6.  When  you  were  saying  the  sentence  with  angry  prosody,  did  you  actually  “feel" 
angry,  or  did  your  voice  only  sound  angry? 
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